openai / openai/codex

Experimental context management: native notes/history return 404 on Pro + Astra, while new_context can discard task state

Open
#43,194 5 comments 4 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug CLI connectivity context memory
Dominant language
Rust
Stars
125k
Forks
19.5k
PR merge metrics
PR metrics pending

Description

Summary

Experimental context management is enabled through the documented opt-in, but the native Codex history/notes endpoints return HTTP 404 for a ChatGPT Pro account, including with gpt-6-astra. Ordinary model requests succeed using the same authorization and account.

This has a serious consequence: the client allows new_context even after saving notes failed. In an earlier persisted task and a separate test, the context window was replaced without successfully preserving/recovering the active task.

We traced the client source, reproduced the HTTP failures, built a temporary local Python backend, and tested specific missing-header hypotheses. We have not identified the private server-side reason for the 404. This report distinguishes observations from hypotheses.

Environment

  • Tested CLI: 0.153.4, bundled with the macOS desktop app.
  • Platform: macOS 26.6.2, arm64 (as reported in the CLI User-Agent).
  • Authentication: ChatGPT sign-in, Pro, not API-key sign-in.
  • Primary supported-model reproduction: gpt-6-astra.
  • Additional comparison: gpt-5.6-luna; its results are not a claim that this experiment officially supports Luna.
  • Normal provider: built-in openai; the original failures used the default https://chatgpt.com/backend-api/codex, without a third-party provider/base-URL override.
  • A custom four-model catalog was active. Its base agent prompts are intentionally empty because custom instructions are supplied separately; token-budget guidance/reminder/fallback fields were restored from the upstream catalog. Per-model model_messages.token_budget.enabled was false, and activation came from the experimental flag. An unmodified-catalog control was not performed.
  • No explicit features.token_budget configuration. use_agent_identity was not enabled.
  • Tests were performed September 6, 2026.

Documented opt-in:

[features.context_management]
experimental_mode = true

The documentation explicitly describes Astra, ChatGPT Plus/Pro, and starting a new task:
https://learn.chatgpt.com/docs/models?surface=app#experimental-context-management

Reproduction without destructive context switching

  1. Sign in with an eligible ChatGPT account and enable the setting above.
  2. Start a fresh task with Astra.
  3. Ask for exactly one direct notes.list_files_by_prefix call with a synthetic prefix and max_results=1, followed by one history.list_windows call with limit=1.
  4. Do not write notes or call new_context for this minimal reproduction.
  5. Inspect the underlying HTTP response, not only the model's final wording.

The tools report:

Unable to perform operation: The backend request failed.

The actual upstream response is:

HTTP 404
Content-Type: application/json
{"detail":"Not found"}

The body is 22 bytes. The automatic alpha/notes/v2/thread_hint request also fails.

Observed native HTTP results

All paths below are under https://chatgpt.com/backend-api/codex.

Model selected in outgoing request Endpoint Status x-oai-request-id
gpt-6-astra alpha/notes/v2/thread_hint 404 b4d85b4e-f4e1-4d20-8335-842d0b4c2f4e
gpt-6-astra responses 200 643a5e67-498b-40a4-b6f1-7f6a9be772b8
gpt-6-astra alpha/notes/v2/list_files_by_prefix 404 1365c86f-5c9d-4a60-8f9d-507fe64fb1b4
gpt-6-astra alpha/history/v2/list_windows 404 90e0314d-f999-4935-9053-a1b0d855339d
gpt-5.6-luna responses 200 3cfa856a-ac7f-4ee0-b155-a11fdfd97519
gpt-5.6-luna alpha/notes/v2/list_files_by_prefix 404 90a33be9-0ce0-4f3e-9740-bd8e56d9b1e4
gpt-5.6-luna alpha/history/v2/list_windows 404 ee01b8a8-c45f-42da-bb76-c6d279a42a58

Both CLI runs exited 0, despite failed tools. Therefore exit status alone is not a success criterion.

Temporary diagnostic backend and transport controls

A Python stdlib server bound only to loopback was used in two modes:

  1. Local mock: the actual CLI received fixture-shaped Responses events invoking the two namespaced functions. Local history/notes routes returned synthetic success markers. Both corresponding function_call_output values reached the next model request. This passed; it proves request dispatch and successful response handling, not remote persistence.
  2. Transparent forwarding: fresh Astra and Luna CLI tasks used a process-only openai_base_url override ending in /backend-api/codex. Requests were forwarded to the same real OpenAI paths with their original bodies and authorization. No canonical configuration/auth/catalog files were changed for these tests.

Request compression was disabled only for these diagnostic processes. The local server returned 426 to the WebSocket handshake to select the CLI's native HTTP fallback; that local 426 is intentional and is not an OpenAI failure. The upstream model requests returned 200. Upstream response headers and stream bytes were relayed, excluding HTTP hop-by-hop/framing headers.

These native CLI probes were ephemeral; the earlier persisted-task failures described below were separate. No account credentials or conversation bodies are included in this report.

Values verified in the captured requests

For both models' notes/history list calls:

  • Authorization was present and equal to the successful /responses authorization.
  • ChatGPT-Account-ID was present and equal to the successful model request's account.
  • JSON context.session_id equaled the model request's Codex metadata session_id.
  • context.current_agent_name equaled metadata agent_name.
  • client_metadata["x-codex-turn-metadata"] contained history_ingest_requested: true.

Thus the history-ingestion opt-in and matching session identity were actually transmitted, not merely inferred from configuration.

Missing-header hypotheses tested

After the native runs, the diagnostic server replayed the captured Astra read-only list requests with unchanged bodies and authentication. These are diagnostic replays, not claims that stock CLI sends the added headers.

Modification Notes list History list
Add only the actual Astra x-codex-routing-hint 404 404
Add x-codex-beta-features: context_management,token_budget 404 404
Add the actual model request's contextual headers plus that beta value 404 404

The contextual set present in this run was session-id, thread-id, x-client-request-id, x-codex-window-id, x-codex-turn-metadata, x-codex-turn-state, and x-codex-routing-hint.

All six returned the same {"detail":"Not found"}. The beta value was an explicit hypothesis, not a documented requirement.

Replay request IDs:

  • Notes routing/beta/context: f80687c2-0c0a-4e3b-a09c-0ce9cd44d14f, 3e5032bc-a4e4-4f38-b0c1-a0109c926aec, 94f6760d-7677-9579-b959-c15dcddf7c13.
  • History routing/beta/context: 3824b3a3-1695-46cb-8326-10b79a3906a6, e6741632-54eb-4967-b890-3ceca8d65560, 915a6404-dd28-4e4f-a163-7b1d865529fd.

This does not exclude an undocumented server requirement, but these particular header additions did not fix the failure.

Source findings

Source references below are pinned to the tested release:

  1. HistoryNotesBackend::call: adds context.session_id and current_agent_name, resolves provider/auth, and sends POST to the alpha route. There is no direct model argument. The server could still infer a model from session state.
  2. HistoryNotesExtension: uses the shared session identity and agent path. A failed thread_hint contributes no recovery hint.
  3. Session::responses_metadata: sets history_ingest_requested from use_history_notes_extension.
  4. Responses header construction: model requests receive routing/session/turn metadata not automatically included in the standalone history/notes path. The replay tests above cover the observed relevant differences.
  5. HTTP transport preserves HTTP status, headers and body in TransportError::Http, but backend.rs discards them when mapping to The backend request failed. This makes diagnosis unnecessarily difficult.
  6. new_context handler does not require a successful notes save before requesting a new window. Turn compaction selection selects the token-budget window reset before ordinary summary compaction. The fallback prompt is not an automatic switch to the old summarization path.
  7. Extension tests and backend tests use local wiremock servers. Their success does not establish real production endpoint availability.
Model gating change already in main

https://github.com/openai/codex/pull/43147 adds supports_experimental_context, enables it for bundled Astra, and fixes fresh child-session activation inheritance. That addresses unsupported-model activation, but does not by itself explain the observed Astra 404.

We compared the relevant 0.153.4 and 6af345407d9c2a568da9d01b6c4b81a9e61495c0 backend/auth/provider/HTTP files. The history-notes request-building code was unchanged; the extension change added thread-hint analytics, not a different backend request.

Earlier persisted-task impact

Existing logs also show 404 for notes/write_file, notes/read_file, and history/list_items.

In a separate Luna test on September 6:

  • 08:20:03 UTC: notes write failed, request ID dc8edce0-2ff4-488a-a6b7-d61a5057b1c4.
  • 08:20:05 UTC: the model called new_context; the next thread-hint request also returned 404.
  • 08:20:06 UTC: the CLI replaced the context window.
  • 08:20:15 UTC: the agent no longer continued the test objective and asked what to do with unrelated remaining input.

The replacement event had an empty message, null compaction_response_id, and no test objective marker in the replacement history. This is evidence of a window transition, not successful state preservation. Existing Astra task logs also showed failures around context transitions.

Related symptom: https://github.com/openai/codex/issues/42693 . That issue does not establish the HTTP cause described here. A separate third-party-proxy report, https://github.com/router-for-me/CLIProxyAPI/issues/5547 , has similar statuses but a different deployment and should not be treated as proof of this first-party case.

Expected behavior / request to maintainers

  • On a supported client/account/model, documented native history/notes routes should be usable.
  • If unavailable for this account or deployment, return an actionable capability/access error rather than an opaque generic 404.
  • Preserve safe HTTP status and request ID in diagnostic errors.
  • Avoid discarding the working context when state preservation/recovery is unavailable, or provide a safe fallback.
  • Please investigate the supplied request IDs for endpoint routing, account eligibility, and session/history-ingestion state.

The remaining server-side possibilities are an unavailable route/deployment, account-specific availability, or missing session resources/ingestion. Public client code and the generic 404 body do not distinguish these. We are not claiming that any particular missing header, model, or server component is proven to be the root cause.

The experimental flag has been disabled locally as mitigation. Diagnostic servers were stopped.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with codex-rs/ext/history-notes/src/backend.rs and codex-rs/http-client/src/transport.rs, then inspect the new_context handler and the history-notes extension and backend tests. Reproduce the 404 using the documented context-management setting and local test coverage. Done means failures expose actionable status/request details and context transitions do not discard work when preservation is unavailable.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
api, backend-api-design, cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.