cactus-compute / cactus-compute/cactus
Gemma4 tool-calling emits declaration-format echoes instead of <|tool_call> (function_calls always empty)
- Dominant language
- C++
- Stars
- 6k
- Forks
- 501
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 4
Description
## Summary
For tool prompts, Gemma4 (via cactus) emits the **declaration** format that cactus injects for tool *definitions* (`<|tool>declaration:NAME{...}`, see `gemma_tools.h:69/330`) instead of the **call** format the parser expects (`<|tool_call>...`, `gemma_tools.h:527-530`). As a result `function_calls` is **always empty** — cactus never parses a valid call.
This is **systemic, not specific to multi-tool prompts.**
## Reproduce
```
cactus test --suite llm
```
All three tool cases show `"function_calls": []` and a `<|tool>declaration:...` response:
| Test | Response (truncated) | `function_calls` | suite verdict |
|---|---|---|---|
| TOOL CALL (1 tool) | `<\|tool>declaration:get_weather{...}` | `[]` | PASS* |
| MULTIPLE (2 tools) | `<\|tool>declaration:send_message{...schema echo...}` | `[]` | FAIL |
| TRIPLE (3 tools) | `<\|tool>declaration:send_message<\|"\|> ...` | `[]` | PASS* |
\* The suite's pass/fail only checks whether the right tool **name** appears in the text — it does not require a parsed call. So the 1- and 3-tool cases "pass" despite producing no usable call; the 2-tool case fails that loose check because the echo is more garbled. The actual behavior (declaration echo, no parsed call) is the same in all three.
## Likely root cause (cactus side)
The model is continuing in the `<|tool>declaration:` *definition* syntax cactus injected, rather than switching to the `<|tool_call>` *call* syntax cactus's parser requires. That points to the tool-call prompt/template/instruction in `gemma_tools.h` (how tools are framed and how the model is told to emit calls), not a Gemma model limitation — the model is producing the wrong *kind* of artifact, not a malformed call.
## Not verified
Reference HF Gemma4 with its native tool-call format was not run, so a model/format-fit contribution isn't fully excluded. But the consistent declaration-echo against cactus's **custom** `declaration:` format is the strong signal.
## Scope
Pre-existing; reproduces on `main`/baseline. Unrelated to the KV-compaction / growable-cache work on `kv-compress-keydiff` (found incidentally while running `cactus test --suite llm` during that PR's validation).
Contributor guide
Research direction
Start by running `cactus test --suite llm` and reading the tool framing and parser sections in `gemma_tools.h`, especially lines 69, 330, and 527-530. Trace how Gemma4 receives tool definitions and how responses become `function_calls`; done means the three reported cases produce parsed calls rather than declaration-format echoes, with the suite verifying that behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- ai
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100