NVIDIA / NVIDIA/NemoClaw

[Ubuntu 26.04][Sandbox] destroy cannot reconcile a retained-recovery record when post-create failure happens before pending-create identity is published

Open
#11,418 0 comments 0 reactions 0 assignees View on GitHub
area: sandbox NV QA platform: ubuntu
Dominant language
TypeScript
Stars
22.5k
Forks
3.1k
Avg merge
1d 1h
Merged PRs (30d)
715

Description

## Description

When a sandbox creation fails before its pending-create identity is published, NemoClaw
writes a retained-recovery record but leaves the sandbox registry row without the identity
and lifecycle-generation fields that destroy needs to bind the two together. destroy then
cannot select a matching recovery record, refuses, and prints a message that tells the
user to resolve an identity conflict without naming any way to do so. The record can never
be reconciled through a supported command.

This is NOT a request to let --force bypass the immutable identity fence. That fence is
intentional and should stay. The ask is that a record the product itself wrote in a
half-initialized state be reconcilable through some supported path, or that the refusal
name a concrete remedy.

This is also distinct from the known case where the registry row DOES carry those fields.
In that case destroy prints a different message that names a remedy - make the owning
gateway report the sandbox absent, then rerun destroy - and following it works. That path
was verified working on a separate host during the same test session. The case reported
here is the one where that remedy does not help.

Platform scope: Reproduced on Ubuntu 25.04 x86_64 only; other platforms not tested.
Regression: Unknown - earlier versions not tested for this path.
OpenShell issue: No

## Environment

```text
Device: Ubuntu 25.04 virtual machine, NVIDIA A100
OS: Ubuntu 25.04
Architecture: x86_64
Node.js: v22.16.0
npm: 10.9.2
Docker: Docker version 29.5.2, build 79eb04c
OpenShell CLI: openshell 0.0.106
NemoClaw: nemoclaw v0.0.121
OpenClaw: 2026.7.1
```

## Steps to Reproduce

```bash
1. Write a Dockerfile that builds successfully on top of a NemoClaw sandbox image and
declares the tool-disclosure build contract:

WORKDIR="$(mktemp -d)"
printf '%s\n' \
'FROM ghcr.io/nvidia/nemoclaw/sandbox-base:v0.0.121' \
'ARG NEMOCLAW_TOOL_DISCLOSURE=progressive' \
'ENV NEMOCLAW_TOOL_DISCLOSURE=$NEMOCLAW_TOOL_DISCLOSURE' \
'RUN echo "BASELINE_BUILD_OK"' > "${WORKDIR}/Dockerfile"

2. Onboard from that Dockerfile on a host where sandbox creation fails after the image
builds but before the sandbox is verified. On this host that happened because a GPU
library required at container start was missing:

nemoclaw onboard --from "${WORKDIR}/Dockerfile" --name tp2 --non-interactive \
--fresh --yes

3. Try the documented cleanup:

nemoclaw tp2 destroy --yes

4. Apply the remedy that resolves the other variant of this state - make the owning
gateway report the sandbox absent, then reconcile:

openshell sandbox delete tp2
nemoclaw tp2 destroy --yes

5. Inspect the registry row and the retained recovery record for that sandbox name.
```

## Expected Result

A retained-recovery record the product wrote itself can be reconciled through a supported
command, or the refusal names a concrete action the user can take to resolve the identity
conflict it reports.

## Actual Result

Step 3, verbatim:

Refusing to destroy retained sandbox 'tp2': NemoClaw could not select exactly one
recovery record from the current immutable registry and Docker identities. No sandbox
resources were removed. Resolve the identity conflict, then rerun 'nemoclaw tp2 destroy'.

Step 4: the openshell delete succeeded and the gateway no longer reported the sandbox:

Deleted sandbox tp2

but the reconcile rerun printed the identical refusal:

Refusing to destroy retained sandbox 'tp2': NemoClaw could not select exactly one
recovery record from the current immutable registry and Docker identities. ...

The record remained unresolved. So the remedy that works for the other variant of this
state does not help here.

Step 5: the sandbox registry row for this name exists but carries neither a
pending-create identity nor a lifecycle generation value, while a retained-recovery record
for the same name does exist with reason "retained_after_sandbox_creation_failure" and
identityWasUnavailable false.

For contrast, in the same session on a different host a sandbox whose registry row DID
carry both fields produced a different refusal that named a remedy, and following that
remedy destroyed the sandbox cleanly and returned the unresolved record count to zero.

The only way to clear the affected state was to edit product state files by hand.

## Logs

```text
Not captured beyond the terminal output quoted in Actual Result. The registry row and the
retained-recovery record contents described in step 5 were read directly from the
product's own state files on the test host.
```

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the failure with `nemoclaw onboard --from ...` followed by `nemoclaw destroy --yes`, then inspect the registry row and retained-recovery record in the product state files. Trace the `destroy` reconciliation path and make the product-written half-initialized record recoverable through a supported command, or ensure the refusal names a concrete remedy.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, node.js, typescript
Domain
cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.