SciSharp / SciSharp/LLamaSharp
[BUG]: Not loading cuda backend on laptop
Nobody has claimed this yet.
- Dominant language
- C#
- Stars
- 3.8k
- Forks
- 508
- Avg merge
- 1d 5h
- Merged PRs (30d)
- 12
Description
Description
on my laptop with a gtx 1060 in it, i have the cpu backend and the cuda backend installed. However the native all log does not show it even loading cuda, but instead shows it trying to load vulkan (not installed) and then eventually defaulting to avx2 and using cpu only. It also allows me to offload gpu layers, but never goes into the gpu buffer.
Reproduction Steps
https://pastebin.com/TmsUg95V this is the pastebin log
Environment & Configuration
- Operating system: Windows 10
- .NET runtime version: 7 (and 8 too)
- LLamaSharp version: 19
- CUDA version (if you are using cuda backend): 11
- CPU & GPU device: Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz
GTX 1060 6GB
Known Workarounds
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the linked Pastebin native log and the CUDA, Vulkan, and CPU backend loading behavior in LLamaSharp v19 on Windows with .NET 7 or 8. Reproduce the reported GTX 1060 configuration and determine why CUDA is skipped; done means the CUDA backend loads and GPU layers use the GPU buffer instead of falling back to Vulkan or AVX2 CPU execution.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- ai
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100