localai cannot run the recommended model
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 49.2k
- Forks
- 4.5k
- Avg merge
- 1d 3m
- Merged PRs (30d)
- 239
Description
LocalAI version:
quay.io/go-skynet/local-ai:master-gpu-nvidia-cuda-13 pulled today
Environment, CPU architecture, OS, and Version:
Linux beast.local 7.2.5-200.fc44.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Sep 11 15:11:05 UTC 2026 x86_64 GNU/Linux
Tue Sep 15 21:41:49 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 615.71.09 KMD Version: 615.71.09 CUDA UMD Version: 13.4 |
+-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GeForce RTX 4070 Ti Off | 00000000:05:00.0 Off | N/A |
| 0% 31C P8 6W / 285W | 164MiB / 12282MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
+-----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| 0 N/A N/A 1846 G /usr/bin/gnome-shell 121MiB |
| 0 N/A N/A 3407 G /usr/bin/Xwayland 3MiB |
+-----------------------------------------------------------------------------------------+
Describe the bug
On a fresh install, localai recommends spark-x2.5-4b. I download and run it, and I get
Error: failed to load model with internal loader: could not load model: rpc error: code = Internal desc = Failed to load model: /models/llama-cpp/models/Spark-X2.5-4B-Q4_K_M/Spark-X2.5-4B-Q4_K_M.gguf. Error: llama_model_load: error loading model: unknown model architecture: 'spark2_5'; llama_model_load_from_file_impl: failed to load model; llama_model_load: error loading model: unknown model architecture: 'spark2_5'; llama_model_load_from_file_impl: failed to load model
To Reproduce
Fresh install, install and start the recommended model. Fails consistently
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the fresh-install failure through the internal loader and inspect the /models/llama-cpp/models/Spark-X2.5-4B-Q4_K_M path and the reported spark2_5 architecture error. Compare the recommended spark-x2.5-4b model with the architectures supported by the loader; done means the recommended model starts without a loading error.
Written by the indexing model from the issue text.
Assessment
- Domain
- ai, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100