openai / openai/codex

Astra long-session delivery failure: 499.7M recorded tokens in one day, lost supervision and goal/stop loops

Open
#46,140 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug CLI context hooks model-behavior rate-limits windows-os
Dominant language
Rust
Stars
125k
Forks
19.4k
PR merge metrics
PR metrics pending

Description

Summary

Long-running Codex/Astra supervision failed to deliver a usable workflow-system vertical slice while consuming approximately half a billion recorded tokens in one local day in a single thread. The operator reports approximately 1 billion tokens across two days and roughly 80% of a weekly allowance consumed in a day. Please investigate both the usage accounting and the agent/goal/compaction behavior.

This is submitted at the operator's explicit request. It is not a claim that every recorded token was uncached, billable at the same rate, or provably wasted. The delivery failure is concrete: the latest increment ended with its UI integration and installed end-to-end test still absent, despite extensive backend validation and repeated progress reports.

Related reports found before filing: #44653, #42901, #44920. This report adds a deduplicated session-ledger measurement and instruction-following/goal-hook failure evidence, rather than alleging only a context-size pricing threshold.

Environment

  • Codex CLI: 0.153.4, verified using codex --version; the session metadata also records this version.
  • Subscription: Pro in the recorded rate-limit metadata; exact multiplier not independently verified.
  • Model: operator identifies the affected trial as Astra; this is a long-lived thread with model changes, so not every historical response is attributed to Astra.
  • Platform: Windows, PowerShell; local filesystem, GitHub and database connectors, custom workflow/stop hooks.
  • Session/thread: 01a07b0d-e74e-7601-af20-9caae7e94b71.
  • Original session start: September 7, 2026. Current incident interval: September 16–17.
  • No diagnostic bundle or raw private source/transcript is attached. A sanitized bundle can be prepared if required.
  • This was not reproduced in a clean/no-hooks environment. Custom instructions and hooks are a relevant confounder, not an excuse to omit the failure.

Measured usage, not an estimate

A local streaming audit read the original session JSONL, selected records of type token_usage_record, deduplicated by response_id, and summed payload.usage (not cumulative turn/thread totals). Calendar dates use America/Denver's UTC-06 offset for this interval.

September 16 local day

UTC window: 2026-09-16T06:00:00Z through 2026-09-17T06:00:00Z, end exclusive.

Metric Recorded count
Unique response usage records 3,245
Input tokens 497,877,236
Cached input tokens (subset of input) 488,561,408
Output tokens 1,789,032
Total input + output 499,666,268
Noncached input by subtraction 9,315,828
Tool call records 3,136
Compaction records 39
User-role hook prompt records 13
Hook prompts containing stale-proof wording 3

Cached input is NOT added a second time. These numbers are this thread's recorded usage, not an independently reconciled invoice/account total. The operator's reported502 million yesterday is close in magnitude; other threads and differing accounting windows may explain differences, but that attribution is not established.

For comparison, the same ledger records September7 total226,873,910 and September8 total195,859,058 (422,732,968 combined). September15 total8,594,943. The 1-billion/two-days report is not independently established by these particular adjacent-day figures and should be reconciled against account telemetry rather than silently accepted or dismissed.

The goal tool separately returned14,723,774 cumulative tokensUsed around06:16UTC September17. That counter is plainly not the same measure as summing response input/output usage. Please explain the difference and expose understandable accounting; the agent initially relied on that smaller counter without auditing the ledger.

Operator screenshots show103 plugin calls and44 skill uses, and a large Codex plan-usage spike on September16. Plugin/skill metrics have different scope from the raw tool/response ledger; no equivalence is claimed.

Time spent on detours: measured lower-level accounting

For the September16 local-day window above, I matched tool call_id records to outputs and calculated the union of call-start to output timestamps:

  • All matched tool-call intervals: 272.60 minutes (4h32m36s).
  • Explicit rdc-work proof verification calls: 83 calls;18.44 minutes.
  • Other governance/orientation-classified calls: 716 calls;69.11 minutes.
  • Other test/build/live-harness-classified calls: 554 calls;68.76 minutes.
  • Other calls: 1,783 calls;116.29 minutes.

Classification was by tool-input text, with precedence:

  1. rdc-work.mjs + verify => proof;
  2. rdc-work / rdc_skill / codeflow / work_item / checklist / validator / lessons => governance;
  3. tsc / node --test / test:ui / live-ladder / live.mjs => test;
  4. otherwise other.

These are tool intervals, not total wall-clock labor or model reasoning time. Batched calls may mix useful and administrative work. Category unions are calculated separately, so they must not be assumed additive if overlapping. 18.44 +69.11 is approximately87.55 minutes of classified proof/governance tool time, not a proven87.55 minutes of entirely wasted work. A defensible exact total of “wild goose chase” time needs further semantic interval review. I will not invent an exact waste percentage.

A confirmed local regression/rework interval occurred September17 around06:13–06:15UTC: extracting a shared stage predicate introduced11 failures in a24-test run; it was corrected and the24 then passed. This is avoidable rework but does not account for the whole day's expenditure.

Expected behavior

The user supplied an existing UI design, existing agent integrations, an installed BPMN workflow engine and an approved layered architecture. They asked for a working5–7-stage outcome-to-artifact prototype, with each stage having inputs, processing, outputs and gates.

The agent should:

  • preserve the supplied UI and reuse existing integrations;
  • demonstrate one installed end-to-end path before expanding breadth;
  • keep ongoing check-ins with Claude, explicitly requested to prevent distraction;
  • distinguish source tests, installed functionality and user-visible outcomes;
  • contain cost and escalate scope/dependency discoveries before another prolonged backend-only cycle;
  • obey an explicit stop/handoff instruction even if an older automatic goal continuation remains active.

Actual behavior and identifiable failures

  1. Backend/test expansion ahead of usable flow. The agent accumulated immutable-candidate, review, publication, provenance and recovery machinery while the existing UI had no connected author/review/publish path.
  2. Checklists displaced the outcome. Some checks were only “harness parses”; others were genuine tests/live observations. The transcript repeatedly presented long checklists, growing to60 checks, then a new6-row task. Green narrow checks did not mean the operator could complete the requested journey.
  3. Stale-proof churn. Stop hooks repeatedly invalidated proof after edits and required reruns. This can be correct mechanically but amplified repeated checking without an effective cost/value boundary.
  4. Required co-supervision was not sustained. The operator explicitly requested Claude/Codex co-design, implementation and review. The agent later admitted treating check-ins as a one-time review rather than an ongoing control. No technical blocker had been established.
  5. Late executable-contract defects. Independent review found publication could accept Design/Done contracts unsupported by the actual executing controllers. The subsequent fix itself caused regression/rework.
  6. One-shot publication stuck on known absent configuration. Missing provider configuration was originally detected only after consuming a durable publication claim. This was corrected in source but not installed.
  7. Cross-endpoint visibility assumption. Separate provider deployment and definition reads were initially treated too strongly; asynchronous visibility required correction.
  8. Status endings despite unattended instruction. The agent ended turns with status rather than continuing authorized work, prompting stop-hook cycles.
  9. Stop/handoff conflict. After the operator explicitly ended the trial, the work item was held for takeover and missing requirements stayed open. Stop hooks still demanded the open DoD; an automatic goal-continuation message then reappeared. The agent ultimately marked the goal blocked rather than fabricating completion.
  10. Wrong reporting destination. When asked to file this failure, the agent initially filed in the user's own project repository instead of OpenAI's. The user had to correct it. This issue corrects that destination.
  11. Historical reported symptoms not all independently reproduced: wrong UI versus design, noninteractive surfaces,401s, startup crashes, credential lookup noise, many database warnings, missing conversation/editor/templates, and worktree confusion. Those remain user-reported historical symptoms; this report does not invent root causes.

Relevant timestamped operator interventions (UTC)

  • Sep16 07:16:42: user says prior work spent a week chasing UI problems on the wrong UI and requests callable API test surfaces.
  • Sep16 08:47:46: explicit goal for Claude/Codex co-design, implementation and review together.
  • Sep16 18:26:46: user reports40% of weekly usage consumed and asks what was accomplished.
  • Sep17 06:13:59: user reports80% usage and points at the unproved publication task.
  • Sep17 06:15:39: explicit termination of the trial and Claude handoff request.
  • Sep17 06:23:24: user reports502 million tokens yesterday.
  • Sep17 06:31:11: user reports1 billion tokens in two days and clarifies submission to OpenAI.

The co-design request to handoff spans 21h27m53s elapsed wall time. This includes productive work, waiting and possible inactivity; it is not all labeled wasted or active compute.

Reproduction / investigation path

This is a recorded long-session incident, not a minimized deterministic reproducer:

  1. Resume the identified long-lived thread with the approved implementation goal and repeated instructions for bounded vertical delivery/co-supervision.
  2. Observe repeated compaction, model calls, administrative/proof work and partial implementation without the completed UI path.
  3. Compare per-response usage records with goal counters and account plan usage.
  4. At explicit user stop/handoff, observe custom stop-checklist messages and automatic goal continuation competing with the latest operator instruction.
  5. Inspect why required peer check-ins and the user-visible acceptance path ceased to govern execution.

Please do not assume all custom harness behavior is an OpenAI defect. The request is to investigate the interaction between the model, long-context/compaction behavior, goal continuation, reporting and cost visibility.

Requested action from OpenAI

  • Reconcile this thread's measured usage with plan consumption and the reported1B/two-day total.
  • Explain cached-input treatment and the mismatch between goal counters and response-ledger totals.
  • Investigate supervision/instruction retention: repeated failure to maintain peer check-ins and deliver the bounded acceptance path.
  • Investigate stop/handoff precedence under active goals and custom hooks.
  • Provide a supported secure route for a sanitized diagnostic transcript if needed.
  • Review possible usage adjustment/credit through the appropriate support channel; this issue does not claim a refund entitlement or contain account secrets.

No production credentials, proprietary source files or raw full transcript have been published here.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the identified session JSONL, the token_usage_record deduplication path, the goal tool counter, and the stop/handoff hook behavior described in the reproduction steps. Compare response-ledger usage with plan telemetry, then reproduce the explicit stop against active goal continuation. Done means the accounting mismatch and instruction-precedence behavior are explained with a supported diagnostic path.

Written by the indexing model from the issue text.

Assessment

Tech stack
powershell, rust
Domain
ai, cli, observability
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.