deepinsight / deepinsight/insightface

insightface + joblib Parallel wont work

Open
#1,433 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
29.7k
Forks
6.1k
PR merge metrics
No merged PRs in 30d

Description

when running insightface's vectorizing functions with `joblib.Parallel` jobs, they complete, but dont return (hangs). Found [this joblib issue](https://github.com/joblib/joblib/issues/138), says that it's due to MKL internal bug.

I'm running this script `MXNET_CPU_WORKER_NTHREADS=4 python run.py --fpath ./imgs.txt --parallel 1`, tried to tweak ENV var, but no result.

Please advise how to parallel insightface's vectorizing using shared model memory as in multiprocessing model per CPU consumes a lot of RAM. Thought that joblib `sharedmem` should solve it, but no luck

Source [code here](https://gist.github.com/Novitoll/28c4796de265da8ccd9e7981f00e021e#file-script-py-L60)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.