Field report: no schema_version in rollout files — silent vocabulary change makes third-party analytics read 0 turns for every session
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 125k
- Forks
- 19.4k
- PR merge metrics
- PR metrics pending
Description
Summary
We run a fail-closed session-analytics layer over Codex rollout files (715 sessions/7 days across 3 production repos, multi-vendor orchestration). Between CLI 0.151.x and 0.153.4 the turn-carrying record changed from event_msg/user_message to response_item/message (with input_text/output_text content parts) — with no schema_version marker at any level and no deprecation event.
Our v1 parser (pinned to the old vocabulary) read every newer session as 0 turns, silently. We now handle both shapes and fail loudly on zero-recognition, but the marker would have made the transition detectable in minutes instead of days.
Reproduction
A minimal, fully tokenized repro (two rollout files + a reference old-vocabulary parser + expected/actual table) is here:
https://github.com/oguzhanpisgin/ai-session-analytics/tree/main/benchmarks/codex-silent-zero
A structural specimen of all observed record types (values tokenized) is here:
https://github.com/oguzhanpisgin/ai-session-analytics/tree/main/specimens/codex
Requests
schema_versionmarker per rollout file (or per line), bumped when the record vocabulary changes.- Structured
task_role/verdictfields for rollouts: in our data, 130 of 364 sessions were verification tasks where a substantiated BLOCK verdict is the successful outcome. Naive end-of-session error scoring reported 61% success; role-aware scoring reports ~94% (paired counts and intervals in the linked repo). First-class fields would remove the need for text inference entirely.
Notes
- JSON key order is not semantic and we do not rely on it; the request is about an explicit version discriminator.
- A separate security report will be filed through the coordinated disclosure channel for an unrelated observation (an opaque credential-shaped string in a reasoning record; no live credential matched).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the linked benchmarks/codex-silent-zero reproduction and specimens/codex, then locate the rollout record definitions and serialization paths in the repository. Clarify the versioning and task-role requirements with maintainers before changing them. Done means the requested metadata is emitted and the existing rollout shapes remain distinguishable to consumers.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- cli, data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100