cactus-compute / cactus-compute/cactus

`parse_messages_json` drops a message's `content` when the JSON key order is `content` before `role`

Open Beginner friendly
#757 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
6k
Forks
501
Avg merge
1d 18h
Merged PRs (30d)
4

Description

# Engine bug: `parse_messages_json` drops a message's `content` when the JSON key order is `content` before `role`

**Component:** `cactus-engine` — `cactus::ffi::parse_messages_json`
**File/line:** [`cactus-engine/src/utils.h:1029`](cactus-engine/src/utils.h) (function begins at line 997)
**Version observed:** `dfd3d8b4` (v1.14-90); **confirmed still present at HEAD `3c4d9263` (2026-07-09)**
**Impact:** Silent, data-dependent prompt corruption — no error is surfaced; the model just runs on a truncated prompt.

---

## Summary

`parse_messages_json` is a hand-rolled string scanner. For each message object it locates `"role"`
first, then searches for `"content"` **starting from the end of the role value**. If a message object
serializes `content` *before* `role` (e.g. `{"content":"…","role":"user"}`), the content key is behind
the search start, is never found within the object, and `msg.content` is left **empty**. The message's
text silently never reaches the model.

Because JSON object key order is not semantically significant, any caller that builds the messages
payload from an **unordered map** can emit either order. This surfaced from a Swift `Dictionary`
serialized via `JSONSerialization`, whose key order is unspecified and was observed to vary
call-to-call — so the same request parses correctly or drops content depending on incidental key order.

## Root cause

```cpp
// cactus-engine/src/utils.h @ HEAD 3c4d9263 — parse_messages_json (abridged; obj_start L1013, content_pos L1029)
size_t obj_start = pos; // '{' of this message object
// … compute obj_end (matching '}') …
size_t role_pos = json.find("\"role\"", pos); // searched from obj_start — OK
// … parse role, role_end = closing quote of the role value …
size_t content_pos = json.find("\"content\"", role_end); // ← BUG: anchored to role_end
if (content_pos != std::string::npos && content_pos < obj_end) {
// parse content …
}
```

`role` is searched from `obj_start`, so role is order-independent. Only `content` is anchored to
`role_end`, making it the sole order-fragile field: content-before-role ⇒ `content_pos` lands in the
*next* object (or npos), fails the `< obj_end` bound, and content is dropped.

## Reproduction (deterministic)

Send a single-message completion with content before role:

```json
[{"content":"What is the capital of France?","role":"user"}]
```

Observed: the prompt tokenizes to only the chat-template scaffolding (content absent); the model
responds as if given an empty/near-empty user turn. Swapping the two keys to `{"role":…,"content":…}`
produces the correct full render. Nothing in between changes.

### Compiled & run against the real function (no device/weights needed)

Verified directly against `parse_messages_json` in `cactus-engine/src/utils.h` — checkout `3c4d9263`,
Apple clang 21.0.0, `arm64-apple-darwin`. `utils.h` → `cactus_kernels.h` → ``, so an ARM
host (Apple Silicon) is required. Test:

```cpp
#include "utils.h" // cactus-engine/src/utils.h
#include
int main() {
std::vector imgs;
auto a = cactus::ffi::parse_messages_json(R"([{"role":"user","content":"hello"}])", imgs);
auto b = cactus::ffi::parse_messages_json(R"([{"content":"hello","role":"user"}])", imgs);
std::cout << "role-first role=[" << a[0].role << "] content=[" << a[0].content << "]\n";
std::cout << "content-first role=[" << b[0].role << "] content=[" << b[0].content << "]\n";
bool ok = (a[0].content == "hello") && (b[0].content == "hello");
std::cout << (ok ? "PASS: both orderings parse content"
: "FAIL: content-first dropped content (bug present)") << "\n";
return ok ? 0 : 1;
}
```

Compile + run from the repo root. At this HEAD the include set is wider than a bare `utils.h` suggests:
`utils.h` → `gemma_tools.h` pulls in vendored `libs/picojson.h`, and `cactus_kernels.h` pulls in
`src/threading.h`, so `-I cactus-engine/libs`, `-I cactus-kernels/src`, and `-I cactus-graph/src` are all
required in addition to the obvious three:

```
$ clang++ -std=c++20 \
-I cactus-engine/src -I cactus-engine/libs \
-I cactus-kernels -I cactus-kernels/src \
-I cactus-graph -I cactus-graph/src \
parse_test.cpp -o parse_test && ./parse_test
role-first role=[user] content=[hello]
content-first role=[user] content=[]
FAIL: content-first dropped content (bug present)
```

Only key position differs between the two payloads; content-first drops the content to empty. Note the
function lives in namespace `cactus::ffi` (call it `cactus::ffi::parse_messages_json`, or add
`using namespace cactus::ffi;`). Confirmed on `3c4d9263` (function at `utils.h:997`, content anchor at
`utils.h:1029`).

## Evidence (controlled A/B, Test app, iPhone 13, dfd3d8b4)

Three input-ordering arms × two models × 4 submits, identical prompt. `msg_sha` = SHA-256 of the exact
bytes sent to the FFI:

| Key order | msg_sha | Qwen3-1.7B | Gemma-4-2B |
|---|---|---|---|
| `role` first (correct) | `a3772c56` (constant) | full structured output, 4/4 | full structured output, 4/4 |
| `content` first (forced) | `70cb9d90` (constant) | content dropped, garbage, 4/4 | content dropped, wrong reply, 4/4 |
| `JSONSerialization` (unordered) | cycles `4c9ca478`/`2432f4e8`/`aefe369a` | output varies per submit | output varies per submit |

`msg_sha` is model-independent (identical bytes → identical hash on both models), and each hash maps to
a fixed output regardless of model — i.e. the output is a pure function of the input bytes, and the only
thing changing is JSON key order. The `content`-first arm fails 100% of the time, deterministically.

## Workaround for callers (until fixed)

Serialize messages with `role` before `content` explicitly (do **not** rely on map order, and note
`sort_keys`/`.sortedKeys` makes it *worse* — `content` sorts before `role`). The test app now
builds the messages JSON by hand (`roleFirstMessagesJSON`) for this reason.

Contributor guide

Open the contributing guide

Research direction

Start in cactus-engine/src/utils.h at cactus::ffi::parse_messages_json, which begins around line 997 and anchors the content search around line 1029. Run the supplied two-payload reproduction with the listed include paths on an ARM host, then verify that both role/content key orders preserve the message content and add regression coverage for that behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend-api-design
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.