[Linux][Upgrade] stopped sandboxes are started for backup but remain SSH-unreachable and block strict upgrade
- Dominant language
- TypeScript
- Stars
- 22.5k
- Forks
- 3.1k
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 715
Description
## Description
On NemoClaw v0.0.126, a strict pre-upgrade backup fails when the inventory contains stopped sandboxes. The installer starts each stopped sandbox, attempts the backup, returns it to the stopped state, and then reports that its in-sandbox SSH endpoint did not answer. The strict backup policy consequently aborts the upgrade.
Platform scope: Reproduced on Ubuntu 24.04 x86_64 in v0.0.126. The same behavior was previously reproduced on Ubuntu 22.04 and Ubuntu 26.04 GPU in v0.0.124.
Regression: This is the stopped-sandbox backup behavior previously addressed by #6500 / #6723, but it remains reproducible in current releases.
OpenShell issue: No evidence. NemoClaw owns the pre-upgrade backup orchestration and readiness timing.
## Environment
```text
NemoClaw: v0.0.126
Platform: Ubuntu 24.04 x86_64
Scenario: two stopped registered sandboxes and one healthy sandbox
```
## Steps to Reproduce
1. Create multiple managed sandboxes.
2. Stop two sandboxes and leave one sandbox running and healthy.
3. Run the supported NemoClaw upgrade installer.
4. Observe pre-upgrade backup processing.
## Expected Result
Stopped sandboxes are either started into a backup-ready state and backed up successfully, or handled according to a documented stopped-sandbox policy without being misreported as running-but-SSH-unreachable.
## Actual Result
Each stopped sandbox is started, its backup fails because the in-sandbox SSH endpoint does not answer, and it is returned to the stopped state. The healthy sandbox is backed up successfully. The installer reports:
```text
Pre-upgrade backup: 1 backed up, 2 failed
```
The strict pre-upgrade backup then aborts the upgrade.
## Reproducibility
Reproduced in a clean targeted v0.0.126 run. The sandbox state restoration assertion passes, while the strict backup success assertion fails consistently.
## Related Issues
- #6500 documents the earlier stopped-sandbox strict-backup failure.
- #6723 implemented the prior fix.
Contributor guide
Research direction
Start at the supported NemoClaw upgrade installer and its pre-upgrade backup processing, using the stopped-sandbox reproduction with two stopped sandboxes and one healthy sandbox. Compare the current behavior with the prior fixes referenced in #6500 and #6723. Done means stopped sandboxes are backed up successfully or handled by a documented policy, strict upgrades no longer fail incorrectly, and sandbox state restoration still passes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- infrastructure, release
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100