OpenClaw tool turns return correct results but fail replay validation and exit 1
- Dominant language
- TypeScript
- Stars
- 22.5k
- Forks
- 3.1k
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 715
Description
## Severity
P1
## Environment
- NemoClaw v0.0.124
- OpenShell 0.0.116
- OpenClaw 2026.7.1
- OpenRouter with a supported Nemotron model
## Reproducer
1. Onboard a working OpenClaw sandbox.
2. Use the named-sandbox agent command to ask OpenClaw to use its exec tool to calculate 7*8 with Python and reply only with the result.
3. Repeat with a plain-text request that does not invoke a tool as a control.
4. Record the payload, structured status, replay metadata, stderr classification, and process exit code.
## Observed behavior
The tool executes and the response payload is 56 with status ok, but the host command exits 1 because replayInvalid=true is detected. The output is classified as a partial trace whose side effects may already have applied. Plain-text turns exit 0.
This creates an ambiguous result: successful side effects and the correct answer are delivered under a failed transcript verdict.
## Expected behavior
A successfully completed tool turn has a replay-valid assistant/tool transcript and exits 0. If transcript integrity cannot be established, the command must fail without presenting the turn as an unambiguous success and must retain the side-effect warning.
## Acceptance criteria
- [ ] A deterministic regression test reproduces a completed OpenClaw exec tool call followed by a final answer.
- [ ] The transcript contains the complete assistant tool call, matching tool result, and final assistant response in replayable order.
- [ ] A completed valid tool turn returns the answer once and exits 0.
- [ ] A genuinely incomplete or invalid replay exits nonzero and warns that side effects may already have applied.
- [ ] Plain-text turn behavior remains unchanged.
- [ ] Tests cover structured JSON passthrough and user-facing stderr classification.
- [ ] Diagnostics expose no credentials, private endpoints, sandbox identities, or sensitive tool output.
Contributor guide
Research direction
Start with the named-sandbox agent command and reproduce the OpenClaw exec tool turn while recording structured JSON, replay metadata, stderr classification, and exit status. Use the acceptance criteria as the completion check: valid transcripts exit 0 with one answer, invalid replays warn about possible side effects, plain-text behavior is unchanged, and diagnostics expose no sensitive data.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- ai, security
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100