Multiprocessing. Pool. MaybeEncodingError: Error sending result: '& lt; Multiprocessing. Pool. ExceptionWithTraceback object at 0 x0000022149d84f98 & gt; '. Reason: 'TypeError (" always serialize' _io. BufferedReader "object",) '
In stackoverflow search some answers, but feeling and I'm not very like this
https://stackoverflow.com/questions/48761983/multiprocessing-pool-maybeencodingerror-error-sending-result-reason-typeerro
https://stackoverflow.com/questions/54736710/multiprocessing-pool-maybeencodingerror-typeerrorcannot-serialize-io-buffe? Noredirect=1
Data set here are some of the information, the thesis process_paper author thesis build some information mainly for the characteristics of the following is a data source:
https://www.biendata.xyz/competition/chaindream_nd_task1/
Def pre_process_pubs (pubs, save_path, pl) :
Cca shut=pl. Starmap (process_paper, zip (pubs. The values ()))
Paper_ids=list (pubs. The keys ())
Pubs=dict (zip (paper_ids, cca shut))
Json. Dump (pubs, open (save_path, 'w'), indent=4)
If __name__=="__main__ ':
Pool=MLP. The pool (4)
# training set
_, pub=load_original_train (0, 1)
Pre_process_pubs (pub, './data/preprocess_pubs/cleaned_train_pubs. Json ', pool)
Del pub
# validation set
The pub=json. The load (open ('./data/OAG - v2 - track1/valid/sna_valid_pub json ', 'r', encoding="utf-8"))
Pre_process_pubs (pub, './data/preprocess_pubs/cleaned_test_pubs. Json ', pool)
Del pub
The pool. The close ()
Use process pool for the first time, now don't know where to begin to find problems, hope to answer, have a big
CodePudding user response:
To answer a wave, this is because in the code specified in the baidu translation of text translation, but some translation URL is too long to open it,