deepinsight / deepinsight/insightface

Expose onnxruntime session options (intra_op_num_threads)

Open
#2,558 0 comments 9 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
29.7k
Forks
6.1k
PR merge metrics
No merged PRs in 30d

Description

The `intra_op_num_threads` setting determines the number of threads used for parallelizing the execution within nodes. By default, this value is set to 0, allowing onnxruntime to choose the appropriate number. However, as mentioned in https://github.com/microsoft/onnxruntime/issues/8313, manually setting the number of threads may be necessary to avoid issues related to thread affinity.

When I use the library in a GPU machine with docker image, I get the following logs:

```
Executing node 273, title: Face Analysis Models, class type: FaceAnalysisModels
�[1;31m2024-04-08 08:26:07.167015003 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1129, index: 0, mask: {48, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2024-04-08 08:26:07.167027853 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1130, index: 1, mask: {1, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2024-04-08 08:26:07.168290961 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1144, index: 15, mask: {8, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
...
```

To address this, I suggest adding onnxruntime session options to the `FaceAnalysis` class, which will allow users to manually adjust the relevant options based on their specific hardware setup.

I also found that another library fixed the same issue: https://github.com/dlr-eoc/ukis-csmask/pull/20

Please let me know if there is already way to set `intra_op_num_threads`, thanks!

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.