Conflict with a CUDA-11 PyTorch installation
- Dominant language
- C++
- Stars
- 28.8k
- Forks
- 8.9k
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 54
Description
XGBoost for Python depends on `nvidia-nccl-cu12`, which is for CUDA 12. I have a PyTorch 2.4.0 installation for CUDA 11.8, but when I use distributed mode, PyTorch picks up on the one installed by XGBoost for NCCL and it gives me problems for my environment.
My workaround is to install the CPU-only version of XGBoost. However, I still want to use XGBoost with CUDA support. It'd be nice if I could use it with `nvidia-nccl-cu11` instead. Not sure what the solution could be (maybe optional groups of dependencies for XGBoost, such as a `cu11` one, etc; or a different package). Note this could be a future problem when CUDA 13 comes out as well.
Contributor guide
No contributing guide indexed for this repository
Research direction
No specific file or test is named. Start by reviewing XGBoost's Python dependency packaging and the CUDA-enabled installation path, then reproduce the conflict with PyTorch 2.4.0 on CUDA 11.8 in distributed mode. Done means CUDA-enabled XGBoost can coexist with the CUDA 11.8 PyTorch installation without NCCL conflicts, with the supported installation choice documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- build-system, distributed-systems, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100