cactus-compute / cactus-compute/cactus

parse_messages_json is key-order-sensitive: `{"content":…,"role":…}` silently generates from an empty user turn

Open Beginner friendly
#772 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
6k
Forks
501
Avg merge
1d 18h
Merged PRs (30d)
4

Description

### Summary

`parse_messages_json` (`cactus-engine/src/utils.h`) locates `"content"` by searching **after** the `"role"` key:

```cpp
size_t role_pos = json.find("\"role\"", pos);
...
size_t content_pos = json.find("\"content\"", role_end);
```

JSON object key order is not significant, and several serializers emit keys in non-fixed order. When a client sends `{"content":"...","role":"user"}` (content first), `content_pos` falls outside the message object and the content is **silently dropped** — the engine renders a template-only prompt (9 tokens for gemma-4) and the model replies along the lines of *"Please provide the text you would like me to process."* No error is surfaced.

### Where it bites

Swift's `JSONSerialization` randomizes dictionary key order **per process**, so an iOS app calling `cactus_complete` with a dict-built messages array generates from an empty user turn on roughly half of launches — while working correctly on the other half. Python's `json.dumps` preserves insertion order, which is why Python-driven usage rarely hits this. (Observed on iOS 27 / iPhone 17 Pro at commit `1ace6d78`; the tell in our logs was `prefill_tokens: 9` instead of 20.)

### Repro (no Swift needed)

```python
from cactus.bindings import cactus as C
m = C.cactus_init("")
buf_ok = C.cactus_complete(m, '[{"role":"user","content":"What is 2+2?"}]', {"max_tokens": 32})
buf_bad = C.cactus_complete(m, '[{"content":"What is 2+2?","role":"user"}]', {"max_tokens": 32})
# buf_ok answers "4"; buf_bad answers "Please provide..." with prompt_tokens ≈ template-only
```

(`cactus_complete` above stands for the messages-JSON string path — passing a pre-serialized string, which the Python binding accepts.)

### Suggested fix

Search for `"content"` within the message object's bounds (`obj_start..obj_end`) independently of the role key's position — the object boundaries are already computed a few lines above. The options parser (`parse_inference_options_json`) is order-insensitive already; only the messages parser has this constraint.

Contributor guide

Open the contributing guide

Research direction

Start in cactus-engine/src/utils.h at parse_messages_json and inspect the existing object boundaries around role parsing. Reproduce with the two message key orders shown in the issue, then verify that both preserve the content and produce equivalent prompts rather than an empty user turn.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
api, backend
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.