NVIDIA / NVIDIA/NemoClaw

[Ubuntu 24.04][Sandbox] Hermes sandbox shields status hangs indefinitely with zero output while sandbox is stuck in Provisioning

Open
#10,104 1 comment 0 reactions 0 assignees View on GitHub
area: sandbox integration: hermes NV QA platform: ubuntu
Dominant language
TypeScript
Stars
22.5k
Forks
3.1k
Avg merge
1d 1h
Merged PRs (30d)
715

Description

## Description

`nemoclaw {sandbox} shields status` (or the `nemohermes` alias) hangs indefinitely with zero output — no stdout, no stderr, no prompt — against a Hermes-agent sandbox, even though onboarding reported success. Reproduced 3 times, 100% hang rate.

Corroborating evidence: `nemoclaw {sandbox} status` (a different, non-shields command) returns fine against the same sandbox in ~2 seconds, but shows the sandbox's Phase as "Provisioning", never "Ready" — confirmed across 3 polls spaced 20s apart, roughly 4 minutes after the onboard wizard itself printed "Hermes Agent gateway is healthy" and completed onboard with exit code 0. A sibling OpenClaw sandbox onboarded the same way on the same host reaches "Ready" immediately and its `shields status` returns correctly in ~3 seconds every time.

This is a different symptom from #9750 (fixed by PR #9866 — that addressed a stale timer-lock on `snapshot create`/`snapshot list` after auto-restore) and from a separately-reported DGX Spark process-tree containment lock — this hang happens on a fresh Hermes sandbox with no shields-down/auto-restore cycle involved at all, and produces zero output rather than an error message.

**Platform scope:** Reproduced on Ubuntu 24.04 x86_64, no GPU. Not yet tested on other platforms.
**Regression:** Unknown — earlier versions not tested for this specific scenario.

## Environment

```text
Device: Ubuntu 24.04 x86_64 server, no GPU
OS: Ubuntu 24.04 LTS
Architecture: x86_64
Node.js: v22.23.2
npm: 10.9.8
Docker: Docker version 29.4.0, build 9d7ad9f
OpenShell CLI: openshell 0.0.106 (docker driver)
NemoClaw: nemoclaw v0.0.114
OpenClaw: N/A (Hermes agent sandbox)
```

## Steps to Reproduce

1. Onboard: `nemoclaw onboard --name {sandbox} --agent hermes` — select NVIDIA Endpoints provider, default model, no web search, no messaging channels, default resources, Balanced policy tier with default presets. Onboard completes and prints "Hermes Agent gateway is healthy" then the final "Start chatting" screen (exit code 0, no errors).
2. Run: `nemoclaw {sandbox} status` — succeeds in ~2s but shows `Phase: Provisioning` (not Ready).
3. Run: `nemoclaw {sandbox} shields status` (or the `nemohermes` alias equivalent).
4. For contrast, onboard a sibling OpenClaw sandbox on the same host/gateway the same way: `nemoclaw onboard --name {sandbox2} --agent openclaw`. Run `nemoclaw {sandbox2} status` (shows `Phase: Ready`) and `nemoclaw {sandbox2} shields status`.

## Expected Result

Per the product's own documented Hermes-parity contract, `nemohermes` output should be identical to `nemoclaw` except for command-hint text and workspace path substitution — `shields status` (and the rest of the shields lifecycle) should behave identically on the Hermes sandbox as it does on the OpenClaw sandbox: fast, with clear output.

## Actual Result

```text
$ timeout 60 nemoclaw {sandbox} shields status
EXIT_CODE:124
(no stdout, no stderr — the process produced literally zero output before being killed by the timeout wrapper)
```

Re-tested with the plain `nemoclaw` binary (not the `nemohermes` alias) against the same sandbox — same hang, confirming it is not alias-specific. Reproduced 3 times, 100% hang rate, zero output every time (not even a partial prompt or error).

For comparison, the exact same command shape against the sibling OpenClaw sandbox (`nemoclaw {sandbox2} shields status`) returned in 3 seconds every time it was run during this session (5+ successful fast invocations).

## Logs

```text
nemoclaw {sandbox} status output (returns fine, unlike shields status):
Sandbox-scoped status for '{sandbox}':
Sandbox: {sandbox}
Model: nvidia/nemotron-3-super-120b-a12b
Provider: nvidia-prod
Inference: not probed (Could not probe https://inference.local/v1/models from inside the sandbox.)
Serving process (hermes agent gateway): not checked
Agent: Hermes Agent version not verified (expected v0.19.0)
Update: unable to verify sandbox Hermes Agent version
Run `nemoclaw {sandbox} rebuild` if this sandbox predates the current install

Sandbox:
Phase: Provisioning (never reached "Ready" — confirmed across 3 polls spaced 20s apart, ~4 minutes after onboard completed successfully)
```

Contributor guide

Open the contributing guide

Research direction

Start with the `nemoclaw {sandbox} status` and `nemoclaw {sandbox} shields status` entry points, reproducing the behavior on the Hermes sandbox and comparing it with OpenClaw. Trace the sandbox Provisioning state and shields lifecycle handling. Done means Hermes shields status returns promptly with clear output instead of hanging with zero output.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, node.js, typescript
Domain
cli, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.