BUG: Reloading changed AGENTS.md can overflow the context window and lock an existing session
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 125k
- Forks
- 19.4k
- PR merge metrics
- PR metrics pending
Description
Bug
Automatically reloading changed AGENTS.md instructions can push a previously usable conversation beyond its context limit and leave it unable to continue. The reload appends the whole updated instruction bundle while the original bundle remains in the request.
This report concerns the resulting context-window failure, regardless of whether full-bundle updates are intentional. The optional freeze/diff/threshold policy request has been moved to #43309. Please classify this issue as a bug; adding a user preference is not required to recognize or fix the failure below.
Observed failure
An affected Codex CLI 0.153.4 session on Linux was successfully using a large instruction bundle with a supported large-context model (gpt-6-astra). Saved records show:
| Observation | Recorded value |
|---|---|
| Last successful request input | 539,936 tokens |
| Client-reported context window | 828,400 tokens |
| Usage before the failed turn | Approximately 65% of the reported window |
| Initial AGENTS snapshot | full: true, 1,877,190 characters of instruction text |
| Later AGENTS update | full: false, 1,883,164 characters of instruction text |
| Next request result | context_window_exceeded |
The updated file grew by 5,974 characters; Codex inserted the entire updated bundle, not only that growth. The generated model-visible instruction messages contained both versions.
The error was:
Codex ran out of room in the model's context window. Start a new thread or clear earlier history before retrying.
The failed request did not return a measured input-token count, so no exact post-reload percentage is claimed. The recorded error, rather than a UI percentage, establishes the failure.
Recovery required a separate offline repair of the generated instruction history in an isolated copy. All 17 actual user/assistant messages were preserved. The repaired conversation then completed a live turn at 542,936 input tokens and an independent-process restart at 542,994. The original conversation was preserved unchanged. This supports that the conversation itself still fit; the automatic instruction duplication caused the lockout. Private instruction contents and transcripts are not included here.
Trigger
- Start a conversation with an instruction bundle that fits but occupies more than half of the usable context window. For project instructions, ensure the configured document-size limit permits the bundle to load.
- Complete a short turn successfully.
- Change the instruction file externally.
- Cold-resume the same conversation and send another short message.
- Codex appends the complete updated bundle alongside the original. If the resulting request exceeds the window, the turn fails instead of providing a usable reload/recovery path.
Expected: automatic instruction reload accounts for the resulting request size before submitting it and preserves a usable continuation or recovery path. If the update cannot fit, handle that condition explicitly rather than repeatedly submitting an oversized request. No particular reload strategy or configurable threshold is required by this bug report.
Actual: an automatic update can turn a working conversation into context_window_exceeded, requiring recovery outside the normal continuation flow.
Small, deterministic reproduction of the duplication mechanism
The following test runs the real CLI with a local fake Responses API, without model inference, production credentials, or outside-network access. It isolates the mechanism using a 6,124-byte synthetic project AGENTS.md:
- Start a thread, exit, and cold-resume without changing the file.
- Change one timestamp character,
2026-01-01T00:00:00Z→2026-01-01T00:00:01Z, keeping the file size identical. - Exit/resume again and inspect the serialized request.
Measured on released CLI 0.153.4:
| Request | Copies of unchanged instruction payload | Request bytes |
|---|---|---|
| Initial | 1 | 48,761 |
| Unchanged cold resume | 1 | 49,033 |
| Timestamp-edited cold resume | 2 | 55,874 |
The unchanged resume adds 272 bytes; the edited resume adds 6,841 bytes. Prior assistant replies remain present, so the duplicate does not replace the conversation history. The script asserts payload/timestamp counts and history preservation, not fixed request sizes.
Evidence boundary: this small test proves full-bundle duplication on cold resume. It does not itself trigger a real model context-limit error, validate compaction/recovery behavior, or establish immediate file-watching during every active session. Its HTTP sizes are bytes; its response usage values are intentionally synthetic. The real lockout evidence is the saved incident above.
Requirements: Linux, Python 3 standard library, bubblewrap (bwrap), and a self-contained Codex executable or one runnable using standard system libraries. An npm launcher depending on files in an unmounted home directory will not work.
python3 reproduce.py --codex /absolute/path/to/codex --keep
--keep retains synthetic request bodies and CLI logs; otherwise scratch is removed. The optional --freeze switch tests the proposed implementation in #43309 and is not supported by released 0.153.4.
Complete reproduce.py
#!/usr/bin/env python3
"""Cold-resume AGENTS.md duplication probe. Linux + bubblewrap, no real API.
python3 reproduce.py [--codex /absolute/path/to/codex] [--freeze] [--keep]
--freeze checks the proposed agents_md_reload_policy="freeze" configuration.
An older CLI may silently ignore that key; assertions deliberately expose this.
"""
import argparse
import http.server
import json
import os
from pathlib import Path
import shutil
import subprocess
import sys
import tempfile
import threading
MARKER = "UNCHANGED_AGENTS_PAYLOAD_7c309f"
OLD = "2026-01-01T00:00:00Z"
NEW = "2026-01-01T00:00:01Z"
REPLY = "SYNTHETIC_PREVIOUS_REPLY_a8f194"
def arguments():
p = argparse.ArgumentParser(description=__doc__)
p.add_argument(
"--codex", default="codex", help="Executable on PATH or absolute path"
)
p.add_argument(
"--freeze", action="store_true", help="Assert proposed freeze behavior"
)
p.add_argument(
"--keep", action="store_true", help="Keep synthetic requests and CLI logs"
)
p.add_argument("--inside", action="store_true", help=argparse.SUPPRESS)
return p.parse_args()
def sandbox(args):
bwrap = shutil.which("bwrap")
executable = shutil.which(args.codex)
if not bwrap or not executable:
raise RuntimeError(
"Requires Linux bubblewrap and a Codex executable; refusing unisolated execution"
)
executable = str(Path(executable).resolve())
scratch = Path(tempfile.mkdtemp(prefix="codex-agents-repro-"))
try:
cmd = [
bwrap,
"--unshare-all",
"--die-with-parent",
"--new-session",
"--clearenv",
"--ro-bind",
"/usr",
"/usr",
]
for name in ("/bin", "/lib", "/lib64"):
path = Path(name)
if path.is_symlink():
cmd += ["--symlink", os.readlink(path), name]
elif path.exists():
cmd += ["--ro-bind", name, name]
cmd += [
"--proc",
"/proc",
"--dev",
"/dev",
"--tmpfs",
"/tmp",
"--tmpfs",
"/run",
"--bind",
str(scratch),
"/work",
"--ro-bind",
executable,
"/opt/codex",
"--ro-bind",
str(Path(__file__).resolve()),
"/app/reproduce.py",
"--setenv",
"PATH",
"/usr/bin:/bin:/opt",
"--setenv",
"HOME",
"/work/home",
"--setenv",
"CODEX_HOME",
"/work/codex-home",
"--setenv",
"LANG",
"C.UTF-8",
"--setenv",
"PYTHONDONTWRITEBYTECODE",
"1",
"--chdir",
"/work",
"--",
"/usr/bin/python3",
"/app/reproduce.py",
"--inside",
"--codex",
"/opt/codex",
]
if args.freeze:
cmd.append("--freeze")
# Both the local HTTP fixture and Codex run inside the SAME isolated netns.
# Real HOME/config/credentials are not mounted; only scratch is writable.
return subprocess.run(cmd, timeout=180).returncode
finally:
if args.keep:
print("Synthetic evidence:", scratch, flush=True)
else:
shutil.rmtree(scratch)
def probe(args):
root = Path("/work")
home = root / "home"
codex_home = root / "codex-home"
work = root / "project"
for directory in (home, codex_home, work):
directory.mkdir()
requests = []
class Handler(http.server.BaseHTTPRequestHandler):
def log_message(self, *unused):
pass
def do_POST(self):
raw = self.rfile.read(int(self.headers["Content-Length"]))
if self.headers.get("Content-Encoding"):
self.send_error(400, "Compression disabled by this reproducer")
return
if self.path != "/v1/responses":
self.send_error(404, "Only the fake Responses endpoint exists")
return
request = json.loads(raw)
requests.append((request, len(raw)))
(root / f"request-{len(requests)}.json").write_bytes(raw)
msg = {
"type": "message",
"id": "msg_fixture",
"status": "completed",
"role": "assistant",
"content": [{"type": "output_text", "text": REPLY, "annotations": []}],
}
events = [
{
"type": "response.created",
"response": {
"id": "resp_fixture",
"object": "response",
"status": "in_progress",
"output": [],
},
},
{"type": "response.output_item.added", "output_index": 0, "item": msg},
{
"type": "response.output_text.delta",
"item_id": "msg_fixture",
"output_index": 0,
"content_index": 0,
"delta": REPLY,
},
{"type": "response.output_item.done", "output_index": 0, "item": msg},
{
"type": "response.completed",
"response": {
"id": "resp_fixture",
"object": "response",
"status": "completed",
"output": [msg],
"usage": {
"input_tokens": 100,
"output_tokens": 2,
"total_tokens": 102,
},
},
},
]
payload = "".join(
"event: " + e["type"] + "\ndata: " + json.dumps(e) + "\n\n"
for e in events
).encode()
self.send_response(200)
self.send_header("Content-Type", "text/event-stream")
self.send_header("Content-Length", str(len(payload)))
self.end_headers()
self.wfile.write(payload)
server = http.server.ThreadingHTTPServer(("127.0.0.1", 0), Handler)
threading.Thread(target=server.serve_forever, daemon=True).start()
agents = "# Synthetic context\nGenerated: " + OLD + "\n" + MARKER + "\n"
agents += (
"\n".join(f"Fixture fact {i}: ordinary test content." for i in range(150))
+ "\n"
)
path = work / "AGENTS.md"
path.write_text(agents)
command = [
args.codex,
"exec",
"--ignore-user-config",
"--ignore-rules",
"--skip-git-repo-check",
"--json",
"-m",
"gpt-5.6-sol",
]
config = {
"model_provider": "mock",
"model_providers.mock.name": "Local fixture",
"model_providers.mock.base_url": f"http://127.0.0.1:{server.server_port}/v1",
"model_providers.mock.wire_api": "responses",
"model_providers.mock.requires_openai_auth": False,
"features.enable_request_compression": False,
"skills.include_instructions": False,
"features.shell_snapshot": False,
"analytics.enabled": False,
"feedback.enabled": False,
"otel.exporter": "none",
"otel.trace_exporter": "none",
}
if args.freeze:
config["agents_md_reload_policy"] = "freeze"
for key, value in config.items():
command += ["-c", key + "=" + json.dumps(value)]
def run(label, extra):
before = len(requests)
result = subprocess.run(
command + extra, cwd=work, capture_output=True, text=True, timeout=45
)
(root / (label + ".stdout")).write_text(result.stdout)
(root / (label + ".stderr")).write_text(result.stderr)
if result.returncode:
raise RuntimeError(
f"{label}: CLI exited {result.returncode}: {result.stderr[-2000:]}"
)
assert len(requests) == before + 1, (
f"{label}: expected exactly one fake API request"
)
request, byte_count = requests[-1]
text = json.dumps(request)
counts = {
"payload_copies": text.count(MARKER),
"old_timestamp": text.count(OLD),
"new_timestamp": text.count(NEW),
"prior_reply": text.count(REPLY),
}
row = {"turn": label, **counts, "request_bytes": byte_count}
print(json.dumps(row), flush=True)
return result, row
try:
print(
subprocess.check_output([args.codex, "--version"], text=True).strip(),
flush=True,
)
print(
"Isolated network namespace; synthetic replies only; no real credentials.",
flush=True,
)
first, a = run("initial", ["Reply briefly."])
events = [
json.loads(line)
for line in first.stdout.splitlines()
if line.startswith("{")
]
sid = next(e["thread_id"] for e in events if e.get("type") == "thread.started")
_, b = run("unchanged_resume", ["resume", sid, "Reply briefly."])
edited = agents.replace(OLD, NEW)
assert len(edited.encode()) == len(agents.encode())
assert sum(x != y for x, y in zip(edited, agents)) == 1
path.write_text(edited)
_, c = run("edited_resume", ["resume", sid, "Reply briefly."])
summary = {
"mode": "freeze" if args.freeze else "default",
"agents_bytes": len(agents.encode()),
"changed_characters": 1,
"request_byte_deltas": [
b["request_bytes"] - a["request_bytes"],
c["request_bytes"] - b["request_bytes"],
],
"turns": [a, b, c],
"token_counts": "synthetic, not a measurement",
}
(root / "summary.json").write_text(json.dumps(summary, indent=2) + "\n")
print(json.dumps(summary), flush=True)
assert a["payload_copies"] == b["payload_copies"] == 1
assert a["old_timestamp"] == b["old_timestamp"] == 1
assert a["new_timestamp"] == b["new_timestamp"] == 0
assert b["prior_reply"] == 1 and c["prior_reply"] == 2, (
"Conversation history was lost"
)
if args.freeze:
assert (c["payload_copies"], c["old_timestamp"], c["new_timestamp"]) == (
1,
1,
0,
), (
"Freeze did not preserve the original AGENTS snapshot (old CLI may ignore the option)"
)
else:
assert (c["payload_copies"], c["old_timestamp"], c["new_timestamp"]) == (
2,
1,
1,
), "Default behavior differs: duplication no longer reproduced"
print(
"PASS: "
+ (
"freeze holds the boot snapshot"
if args.freeze
else "duplication reproduced"
),
flush=True,
)
return 0
finally:
server.shutdown()
server.server_close()
if __name__ == "__main__":
args = arguments()
try:
sys.exit(probe(args) if args.inside else sandbox(args))
except (AssertionError, RuntimeError, subprocess.TimeoutExpired) as error:
print("FAIL:", error, file=sys.stderr)
sys.exit(1)
Environment
- Codex CLI 0.153.4, Linux 6.3.7-060307-generic, x86-64.
- Real incident:
gpt-6-astra; client-reported window 828,400 tokens. - Mechanical test:
gpt-5.6-solis request metadata only; no model is called and no subscription is needed. - Duplication also reproduced in default mode with a development build based on
8d7cc24a87f4aa66aa434eb4f25f4f4bafc0e0a9. - Scope of this report: Codex instruction reload and its context-budget failure. Other harnesses and optional reload policies are outside this bug's scope.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running reproduce.py with the released Codex executable and inspect the serialized requests sent to the local /v1/responses fixture. Trace the cold-resume handling for changed AGENTS.md instructions. Done means an updated instruction bundle no longer makes a usable session submit an oversized request, and the failure has an explicit continuation or recovery path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, rust
- Domain
- cli
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100