deepinsight / deepinsight/insightface
insightface + joblib Parallel wont work
- Dominant language
- Python
- Stars
- 29.7k
- Forks
- 6.1k
- PR merge metrics
- No merged PRs in 30d
Description
when running insightface's vectorizing functions with `joblib.Parallel` jobs, they complete, but dont return (hangs). Found [this joblib issue](https://github.com/joblib/joblib/issues/138), says that it's due to MKL internal bug.
I'm running this script `MXNET_CPU_WORKER_NTHREADS=4 python run.py --fpath ./imgs.txt --parallel 1`, tried to tweak ENV var, but no result.
Please advise how to parallel insightface's vectorizing using shared model memory as in multiprocessing model per CPU consumes a lot of RAM. Thought that joblib `sharedmem` should solve it, but no luck
Source [code here](https://gist.github.com/Novitoll/28c4796de265da8ccd9e7981f00e021e#file-script-py-L60)
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.