google-deepmind / google-deepmind/gemma
gemma4:26b concurrent decode loses EOS (length at num_predict); qwen3.8-27b IDENTICAL dual on same GB10/Ollama harness is 3/3 clean #18442
- Dominant language
- Python
- Stars
- 5.7k
- Forks
- 1k
- Avg merge
- 10h 33m
- Merged PRs (30d)
- 2
Description
Two concurrent generations of `gemma4:26b` (26B-A4B-it) on Ollama 0.33.3 / DGX Spark (GB10) often lose EOS. One or both arms run to the client `num_predict` cap (50000) with verbatim repeated reasoning. Solo, same prompt: 8/8 `stop`, 14k–23k tokens.
Same two-call harness on `qwen3.8:27b`, same Ollama process, same box: 6/6 `stop`. Unique leading markers on system and user text do not fix Gemma.
Likely runner batch/KV isolation × this architecture (hybrid SWA + MoE), not a general “Gemma never stops” on this task.
Engine tickets (full tables and repro):
- Ollama: https://github.com/ollama/ollama/issues/18442
- llama.cpp: https://github.com/ggml-org/llama.cpp/issues/28917
Related, not the same bug:
- #610 — this size class, solo list-generation loop
- #727 — 12B multi-turn thinking loop
Contributor guide
Research direction
Start by reproducing the two concurrent gemma4:26b generations with Ollama 0.33.3 on the DGX Spark/GB10 setup, then compare them with solo generation and the qwen3.8:27b control. Read the linked Ollama #18442 and llama.cpp #28917 reports for the full harness and tables; done requires an identified, actionable fix or a confirmed handoff to the affected engine.
Written by the indexing model from the issue text.
Assessment
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100