[DGX Spark][Onboard] managed llama.cpp resume hangs past its own 900s timeout waiting for lifecycle record retirement
- Dominant language
- TypeScript
- Stars
- 22.5k
- Forks
- 3.1k
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 715
Description
## Description
After a managed llama.cpp onboard is interrupted mid-build and then resumed, the resume no longer fails immediately with the previously-reported lifecycle-authority error (that symptom is now fixed), but instead hangs indefinitely in a "Waiting for OpenShell to retire the previous lifecycle record" step, exceeding its own documented cap of roughly 900 seconds with no forward progress. The process never completes and must be forcibly terminated.
Platform scope: Reproduced on DGX Spark only; other platforms not tested.
Regression: Unknown — earlier versions not tested. This specific hang was not reachable in earlier builds because resume used to fail earlier in the same flow with a different error; whether this hang exists in earlier versions under some other trigger path is untested.
OpenShell issue: No
## Environment
```text
Device: DGX Spark
OS: Ubuntu 24.04.4 LTS
Architecture: aarch64
Node.js: v22.23.1
npm: 10.9.8
Docker: Docker version 29.2.1, build a5c7197
OpenShell CLI: openshell 0.0.106
NemoClaw: nemoclaw v0.0.114
OpenClaw: N/A (onboard never reaches Ready in this repro)
```
## Steps to Reproduce
1. On a DGX Spark host running NemoClaw v0.0.114, start a managed llama.cpp onboard:
```bash
NEMOCLAW_EXPERIMENTAL=1 NEMOCLAW_PROVIDER=install-llama-cpp NEMOCLAW_YES=1 nemoclaw onboard --non-interactive --name {sandbox}
```
2. Interrupt with Ctrl+C during the "Building sandbox image..." sub-step inside step [6/8] Creating sandbox.
3. Re-run the exact same onboard command to resume.
4. Observe the resume progress.
## Expected Result
Resume either completes successfully to a Ready sandbox, or fails promptly with a clear error, within its own documented roughly 900 second timeout.
## Actual Result
Resume hangs indefinitely in a "Waiting for OpenShell to retire the previous lifecycle record" step. No forward progress occurs even after the documented timeout window elapses; the process was manually terminated after approximately 19 minutes with no progress or error.
## Logs
```text
Waiting for OpenShell to retire the previous lifecycle record... (up to 900s)
The above line repeated with no further output until manual termination at roughly the 1140 second mark. Full raw terminal capture beyond this line was not preserved.
```
Contributor guide
Research direction
Reproduce the interruption and resume flow using the documented nemoclaw onboard command on a DGX Spark, then trace the resume path at the “Waiting for OpenShell to retire the previous lifecycle record” step. Done means resume either reaches a Ready sandbox or fails clearly within the documented roughly 900-second timeout.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100