awslabs / awslabs/cli-agent-orchestrator
codex: native TUI activity rows leak into extract_last_message_from_script handoff results
- Dominant language
- Python
- Stars
- 1.3k
- Forks
- 267
- Avg merge
- 1d 23h
- Merged PRs (30d)
- 70
Description
## Summary
`CodexProvider.extract_last_message_from_script` filters MCP tool-call markers (`MCP_TOOL_CALL_PATTERN`, `providers/codex.py:48`) but nothing filters Codex's **native** TUI activity rows (`• Explored …`, `• Ran …`, `• Read …`). On the primary extraction path they leak into the extracted response, so a handoff result can arrive with tool-activity noise prepended to the model's actual answer.
Credit where due: this was diagnosed by @dechow73-maker in #235. That PR is being closed as superseded (both of its halves have since landed on `main` independently — see the review there), but this residual gap outlived it and deserves its own tracking.
## Repro (verified on current `main`, f45f322)
```python
from cli_agent_orchestrator.providers.codex import CodexProvider
extract = CodexProvider.extract_last_message_from_script
raw = "› fix the failing test\n\n• Explored src/providers\n• Ran pytest -q\n\n• The bug is in the poll loop.\n"
```
| case | input bullets | output |
|---|---|---|
| MCP call | `• Called myserver.load_skill(name="x")` + `└ ok` + answer | ✅ `'• Done: the skill loaded fine.'` — marker + tool body stripped |
| legit answer starting with a verb | `• Called attention to the flaky test and fixed it.` | ✅ preserved |
| **native TUI activity** | `• Explored src/providers` / `• Ran pytest -q` + answer | ❌ `'• Explored src/providers\n• Ran pytest -q\n\n• The bug is in the poll loop.'` |
## Mechanism
On the primary path (a `›` user line is found), `_find_assistant_marker` (`providers/codex.py:281`) anchors the response at the **first** `•` line after the user message that is not MCP-call-shaped. Native activity rows are `•` lines that don't match `.(`, so the first one becomes `response_start` and everything from there to the boundary is returned (`providers/codex.py:794-799`).
The fallback path (no user line) is unaffected — it anchors on the **last** assistant marker, which is why the leak only shows on real interactive captures.
## Constraint on the fix
A verb allowlist (as #235 proposed via `TUI_ACTIVITY_PATTERN`) would regress a case `main` guards deliberately: `• Called attention to the flaky test and fixed it.` is a legitimate final answer that a verb list (`Called`, `Read`, `Ran`, `Created`, `Updated`, `Applied`, …) would swallow — `codex.py:45-47` names exactly this false positive as the reason the MCP filter keys on the *shape* `.(` instead. Any fix here needs the same discipline: key on structural shape (e.g. the `└`-prefixed continuation body, status-row layout, or footer adjacency), not on English verbs.
Contributor guide
Research direction
Start in providers/codex.py with CodexProvider.extract_last_message_from_script and _find_assistant_marker, then reproduce the interactive capture shown in the issue. The fix is done when native TUI activity rows are excluded from the primary extraction path, legitimate bullet answers remain intact, and the unaffected fallback path still behaves as described.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli, tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 62/100