huggingface / huggingface/candle
Support for Nvidia unified memory?
- Dominant language
- Rust
- Stars
- 21k
- Forks
- 1.8k
- Avg merge
- 16h 42m
- Merged PRs (30d)
- 25
Description
All,
I saw this morning that Tim Dettmers bitsandbytes python lib uses Nvidia's [Unified Memory](https://developer.nvidia.com/blog/unified-memory-cuda-beginners/) by [default](https://x.com/stasbekman/status/1749968490155696612), see (`csrc/pythonInterface.c:377`).
It doesn't look like candle, via cudarc supports this.
I'm interested in supporting this feature in Rust because my current GPU has a decent compute score, but only 4GB of VRAM (T1000).
Is there any plan to support this in candle?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading the bitsandbytes reference at csrc/pythonInterface.c:377 and tracing how candle currently reaches cudarc. Determine whether cudarc can expose Nvidia Unified Memory and what candle integration would require; done means a decided implementation path and verified support for the reported low-VRAM use case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, rust
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100