openai / openai/codex

Agent behaviour: session consumed ~42% of a weekly context allowance to deliver a small scoped change

Open
#46,343 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug model-behavior rate-limits subagent
Dominant language
Rust
Stars
125k
Forks
19.4k
PR merge metrics
PR metrics pending

Description

Summary

A single scoped implementation task was completed successfully: code delivered, full test suite
green, live hardware-in-the-loop trials passed, independent review returned pass with residual
risk, branch merged.

The problem is cost, not outcome. The session consumed roughly 42% of a weekly context
allowance. The final diff was small — about thirty files and low four figures of added characters.
That ratio is the defect, and several of the contributing behaviours required repeated operator
intervention to stop.

Reported as model/agent behaviour. The session ran through an agent harness against the model,
so the specific CLI is incidental; the failure patterns below are about how the agent spent
context and operator time, not about any one wrapper. Project specifics are deliberately
omitted — the work is private.

Behaviours that consumed the budget

  1. Repeated enumeration of the same introspection surface. The same metadata listing was
    swept five or more times, at least once returning output identical to an earlier pass. Each
    pass pulled hundreds of rows into context. What was actually needed was a short candidate
    list obtainable in one pass.

  2. Searching reference material that could not be authoritative. Time was spent globbing
    and grepping vendored third-party reference trees which the project's own contract already
    designates as dated and non-authoritative. No result there could have unblocked the decision.

  3. Operator left idle during agent-only work. On three separate occasions the operator was
    held waiting in a live session while the agent performed work that required no live
    environment. The operator raised this three times before the behaviour stopped.

  4. Blocking delegation issued in the same turn as an operator request. Twice, the agent
    asked the operator to perform a manual action and then immediately started a synchronous
    sub-agent call, so the operator's reply could not be acknowledged until the delegate
    returned. The operator had to ask whether the agent was running in the foreground.

  5. Silent no-op delegations. Three sub-agent invocations were issued with empty or
    cancelled arguments, returned nothing, and were assumed to have succeeded. Version control
    showed no changes for several subsequent steps.

  6. Advisory findings read but not acted on. An automated shadow reviewer surfaced correct
    blockers, including an explicit "stop the discovery loop" and two correctness warnings that
    later proved accurate. They were repeatedly acknowledged without disposition. The operator
    had to escalate twice, ending in an explicit instruction, before each finding was resolved
    before the next action.

  7. Raw measurement transcripts staged into the repository. Multi-megabyte per-frame captures
    were written into the working tree and left staged for commit. The operator caught it. They
    were removed and replaced with a small aggregate summary carrying the same conclusions. The
    captures were roughly two orders of magnitude larger than the entire real change.

  8. A preventable second hardware cycle. A low-level field was read at the wrong width
    because the width was assumed rather than confirmed from metadata that was already
    available. The first live trial returned garbage, costing the operator a full shutdown,
    rebuild, redeploy, relaunch and manual re-setup cycle.

  9. Instrument churn on an unverified hypothesis. A test harness threshold was changed to fix
    a symptom before reading the component that produced it. The change was wrong and was
    reverted once the source was read; the correct fix was an existing flag.

  10. A documentation edit destroyed a prior accepted record. Updating a status entry
    overwrote a previously accepted historical fact instead of appending alongside it. Caught
    and restored before commit.

Guardrails that would have prevented most of this

  • Hard stop on discovery loops. After a bounded number of enumeration or search passes
    against the same surface with no new decision, require an explicit stop: commit to the
    available evidence, or record the value as unavailable. "One more sweep" was the dominant
    failure mode of this session.
  • Requesting an operator action must end the turn. Never start a blocking call while
    awaiting a human reply.
  • Treat a delegation that returns no file changes as failed, not complete.
  • Make advisory/reviewer findings blocking by default — apply, verify already satisfied, or
    state an evidence conflict, before the next action.
  • Raw measurement output must never enter the repository or the conversation. Derive the
    aggregate; discard the transcript.
  • Confirm low-level field widths from available metadata before writing the read. A wrong
    width costs a human-in-the-loop cycle, not just a failed test.

Usage and compensation

I am asking for the consumed allowance to be investigated and restored. Roughly 42% of a weekly
context budget went on a task whose delivered output was a small diff, and a substantial share
of that spend is directly attributable to the behaviours above: redundant enumeration passes,
megabytes of raw transcript pulled into context, repeated re-reading of material already read,
and rework caused by assumptions the agent could have checked. Those are not user error. The
operator repeatedly and explicitly intervened to stop them, and the agent continued until
instructed in the strongest terms.

If this repository is not the correct venue for a usage adjustment, please redirect it to the
right channel rather than closing it — but the behavioural findings above stand on their own and
are worth triaging regardless of the billing question.

Out of scope

The delivered work itself is sound and independently reviewed. This issue concerns session cost
and the agent behaviours that produced it. Project identifiers, paths, symbols and domain
details are intentionally excluded.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue names no files, tests, symbols, or entry points, and deliberately omits project-specific details. Start by locating the agent session, delegation, discovery, measurement, and review-handling components, then establish reproducible tests for the listed behaviors. Done means the relevant guardrails are implemented, verified, and the reported context and operator-cost regressions are addressed.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
ai-infra-agents, devtools
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.