macaroni-os / macaroni-os/mark-issues
[ai-kit] Review ollama ebuild to better support NVIDIA GPU
Nobody has claimed this yet.
- Dominant language
- No language data
- Stars
- 5
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
In the last releases of ollama the CUDA driver uses the new dev-util/nvidia-cuda-toolkit package with different releases:
- CUDA 13
- CUDA 12
- CUDA 11 (to test)
Every release supplies support for different devices:
if(CUDA_VERSION VERSION_GREATER_EQUAL "13.0")
# CUDA 13:
set(CMAKE_CUDA_ARCHITECTURES "75;80;86;89;90;100;120")
elseif(CUDA_VERSION VERSION_GREATER_EQUAL "12.0")
# CUDA 12:
set(CMAKE_CUDA_ARCHITECTURES "61;70;75;80;86;89;90")
else()
# CUDA 11: Max Ampere (86), include Maxwell (50)
set(CMAKE_CUDA_ARCHITECTURES "50;52;61;70;75;80;86")
endif()
What is saw is that the native detection of CUDA GPU currently used in the ollama ebuild doesn't work correctly every time.
Only forcing CMAKE_CUDA_ARCHITECTURES with value 75 permit to have my GeForce RTX 2060 Mobile working correctly.
So what we need to do is:
a) permit to customize CUDA_ARCHITECTURES from users based on their GPUs. Keep them the control is better.
b) try to use the reported values on CMAKE_CUDA_ARCHITECTURES based on the available nvidia-cuda-toolkit package or just keep just point a
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Inspect the ollama ebuild and its current CUDA GPU detection, then compare the requested CUDA 11, 12, and 13 architecture values with the available nvidia-cuda-toolkit releases. The work is done when users can customize CUDA_ARCHITECTURES and the ebuild handles the supported toolkit versions without relying on unreliable native detection.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cmake, ollama
- Domain
- ai, build-system
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 45/100