python 多进程调用失败
- Dominant language
- C++
- Stars
- 4k
- Forks
- 592
- PR merge metrics
- No merged PRs in 30d
Description
```python3
# SentenceAnalysis() ## 这里做实例化lac的模型
pool=Pool(processes=30)
res= pool.map(SentenceAnalysis().sentence_struct,querylist) ## sentence_struct就是用的lac来iqu词性、词的重要性等操作
```
Traceback (most recent call last):
File "/mnt/data/yzy/project/DataPrepare/bin/VoiceDemand.py", line 199, in
res= pool.map(sentenceanaly.sentence_struct,querylist)
File "/home/yiziyang/.conda/envs/nlu_train/lib/python3.7/multiprocessing/pool.py", line 268, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "/home/yiziyang/.conda/envs/nlu_train/lib/python3.7/multiprocessing/pool.py", line 657, in get
raise self._value
File "/home/yiziyang/.conda/envs/nlu_train/lib/python3.7/multiprocessing/pool.py", line 431, in _handle_tasks
put(task)
File "/home/yiziyang/.conda/envs/nlu_train/lib/python3.7/multiprocessing/connection.py", line 206, in send
self._send_bytes(_ForkingPickler.dumps(obj))
File "/home/yiziyang/.conda/envs/nlu_train/lib/python3.7/multiprocessing/reduction.py", line 51, in dumps
cls(buf, protocol).dump(obj)
TypeError: can't pickle paddle.fluid.core_avx.CPUPlace objects
Contributor guide
No contributing guide indexed for this repository
Research direction
Reproduce the failure from VoiceDemand.py line 199, focusing on multiprocessing.Pool.map with SentenceAnalysis().sentence_struct and the reported paddle.fluid.core_avx.CPUPlace pickling error. Trace how the LAC model is instantiated and determine a supported multiprocessing usage; done means the call processes querylist without the serialization error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100