openai / openai/codex

Field report: no schema_version in rollout files — silent vocabulary change makes third-party analytics read 0 turns for every session

Open
#43,593 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

CLI enhancement session
Dominant language
Rust
Stars
125k
Forks
19.4k
PR merge metrics
PR metrics pending

Description

Summary

We run a fail-closed session-analytics layer over Codex rollout files (715 sessions/7 days across 3 production repos, multi-vendor orchestration). Between CLI 0.151.x and 0.153.4 the turn-carrying record changed from event_msg/user_message to response_item/message (with input_text/output_text content parts) — with no schema_version marker at any level and no deprecation event.

Our v1 parser (pinned to the old vocabulary) read every newer session as 0 turns, silently. We now handle both shapes and fail loudly on zero-recognition, but the marker would have made the transition detectable in minutes instead of days.

Reproduction

A minimal, fully tokenized repro (two rollout files + a reference old-vocabulary parser + expected/actual table) is here:
https://github.com/oguzhanpisgin/ai-session-analytics/tree/main/benchmarks/codex-silent-zero

A structural specimen of all observed record types (values tokenized) is here:
https://github.com/oguzhanpisgin/ai-session-analytics/tree/main/specimens/codex

Requests

  1. schema_version marker per rollout file (or per line), bumped when the record vocabulary changes.
  2. Structured task_role / verdict fields for rollouts: in our data, 130 of 364 sessions were verification tasks where a substantiated BLOCK verdict is the successful outcome. Naive end-of-session error scoring reported 61% success; role-aware scoring reports ~94% (paired counts and intervals in the linked repo). First-class fields would remove the need for text inference entirely.

Notes

  • JSON key order is not semantic and we do not rely on it; the request is about an explicit version discriminator.
  • A separate security report will be filed through the coordinated disclosure channel for an unrelated observation (an opaque credential-shaped string in a reasoning record; no live credential matched).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the linked benchmarks/codex-silent-zero reproduction and specimens/codex, then locate the rollout record definitions and serialization paths in the repository. Clarify the versioning and task-role requirements with maintainers before changing them. Done means the requested metadata is emitted and the existing rollout shapes remain distinguishable to consumers.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
cli, data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.