google-deepmind / google-deepmind/gemma

gemma4:26b concurrent decode loses EOS (length at num_predict); qwen3.8-27b IDENTICAL dual on same GB10/Ollama harness is 3/3 clean #18442

Open
#798 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
5.7k
Forks
1k
Avg merge
10h 33m
Merged PRs (30d)
2

Description

Two concurrent generations of `gemma4:26b` (26B-A4B-it) on Ollama 0.33.3 / DGX Spark (GB10) often lose EOS. One or both arms run to the client `num_predict` cap (50000) with verbatim repeated reasoning. Solo, same prompt: 8/8 `stop`, 14k–23k tokens.

Same two-call harness on `qwen3.8:27b`, same Ollama process, same box: 6/6 `stop`. Unique leading markers on system and user text do not fix Gemma.

Likely runner batch/KV isolation × this architecture (hybrid SWA + MoE), not a general “Gemma never stops” on this task.

Engine tickets (full tables and repro):
- Ollama: https://github.com/ollama/ollama/issues/18442
- llama.cpp: https://github.com/ggml-org/llama.cpp/issues/28917

Related, not the same bug:
- #610 — this size class, solo list-generation loop
- #727 — 12B multi-turn thinking loop

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the two concurrent gemma4:26b generations with Ollama 0.33.3 on the DGX Spark/GB10 setup, then compare them with solo generation and the qwen3.8:27b control. Read the linked Ollama #18442 and llama.cpp #28917 reports for the full harness and tables; done requires an identified, actionable fix or a confirmed handoff to the affected engine.

Written by the indexing model from the issue text.

Assessment

Domain
machine-learning, performance
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.