deepinsight / deepinsight/insightface

Error in train phase (libnccl.so.2: cannot open shared object file: no such file or directory)

Open
#2,077 12 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
29.7k
Forks
6.1k
PR merge metrics
No merged PRs in 30d

Description

Hello.
I want to train arcface_mxnet recognition model. I installed cuda10.0 and mxnet-cu100 and got this error after running train.py:
![image](https://user-images.githubusercontent.com/10956392/184878004-a3fcda5a-1ba6-42f6-ac63-45bcc1e899d9.png)
By the way, I try cuda10.2 and mxnet-cu102 but I got following error too:
**OSError: libcudnn.so.8: cannot open shared object file: No such file or directory**
I appreciate your help.
So many thanks in advance.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with train.py and reproduce the training failure in the reported environment. Check how the arcface_mxnet training entry point loads its CUDA and MXNet dependencies, including the missing libnccl.so.2 and libcudnn.so.8 libraries. Done means training starts without either missing shared-library error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.