NVIDIA / NVIDIA/NemoClaw

OpenClaw tool turns return correct results but fail replay validation and exit 1

Open
#11,844 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
22.5k
Forks
3.1k
Avg merge
1d 1h
Merged PRs (30d)
715

Description

## Severity

P1

## Environment

- NemoClaw v0.0.124
- OpenShell 0.0.116
- OpenClaw 2026.7.1
- OpenRouter with a supported Nemotron model

## Reproducer

1. Onboard a working OpenClaw sandbox.
2. Use the named-sandbox agent command to ask OpenClaw to use its exec tool to calculate 7*8 with Python and reply only with the result.
3. Repeat with a plain-text request that does not invoke a tool as a control.
4. Record the payload, structured status, replay metadata, stderr classification, and process exit code.

## Observed behavior

The tool executes and the response payload is 56 with status ok, but the host command exits 1 because replayInvalid=true is detected. The output is classified as a partial trace whose side effects may already have applied. Plain-text turns exit 0.

This creates an ambiguous result: successful side effects and the correct answer are delivered under a failed transcript verdict.

## Expected behavior

A successfully completed tool turn has a replay-valid assistant/tool transcript and exits 0. If transcript integrity cannot be established, the command must fail without presenting the turn as an unambiguous success and must retain the side-effect warning.

## Acceptance criteria

- [ ] A deterministic regression test reproduces a completed OpenClaw exec tool call followed by a final answer.
- [ ] The transcript contains the complete assistant tool call, matching tool result, and final assistant response in replayable order.
- [ ] A completed valid tool turn returns the answer once and exits 0.
- [ ] A genuinely incomplete or invalid replay exits nonzero and warns that side effects may already have applied.
- [ ] Plain-text turn behavior remains unchanged.
- [ ] Tests cover structured JSON passthrough and user-facing stderr classification.
- [ ] Diagnostics expose no credentials, private endpoints, sandbox identities, or sensitive tool output.

Contributor guide

Open the contributing guide

Research direction

Start with the named-sandbox agent command and reproduce the OpenClaw exec tool turn while recording structured JSON, replay metadata, stderr classification, and exit status. Use the acceptance criteria as the completion check: valid transcripts exit 0 with one answer, invalid replays warn about possible side effects, plain-text behavior is unchanged, and diagnostics expose no sensitive data.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
ai, security
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.