Hermes GPU fallback refuses an exact managed cleanup handoff
- Dominant language
- TypeScript
- Stars
- 22.5k
- Forks
- 3.1k
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 715
Description
## Investigation summary
PR #9323's exact unfiltered E2E run at commit `5edd54a44cdc124b5ecf06ff3cda3e134803e857` fails only the Hermes native-to-compatibility fallback scenario. Native GPU startup and compatibility-only startup both pass.
The fallback failure is distinct from #9486. The #9512 repair is present, no shared-state metadata mismatch or duplicate managed container remains, and strict cleanup destroys the sandbox and releases the gateway port.
## Root cause
Commit `d33393958` added `cleanupNativeGpuFailureForFallback()`. When the managed lifecycle returns an exact owner-bound cleanup handoff, the helper returns `safe: false`. The caller then refuses the compatibility retry even though the exact managed cleanup path completed safely.
The prior candidate `c073313` contains #9512 and passes the same fallback scenario. The regression begins with the later #9323 cleanup change.
## Evidence
- Failing exact run: https://github.com/NVIDIA/NemoClaw/actions/runs/32512936706/job/96868526146
- Passing native scenario at the same commit: https://github.com/NVIDIA/NemoClaw/actions/runs/32512936706/job/96868526030
- Passing compatibility-only scenario at the same commit: https://github.com/NVIDIA/NemoClaw/actions/runs/32512936706/job/96868526154
- Passing fallback scenario before the regression: https://github.com/NVIDIA/NemoClaw/actions/runs/32412828260/job/96584043003
## Accepted scope
- Preserve exact managed lifecycle ownership and fail-closed cleanup.
- Treat an exact owner-bound cleanup handoff that completes and reconciles external state as safe for the one intended compatibility retry.
- Do not fall back to mutable sandbox-name container selection.
- Do not add a broad retry or weaken strict cleanup.
- Add deterministic coverage for the exact cleanup-handoff result and retry decision.
## Acceptance evidence
- Focused lifecycle and Hermes GPU fallback tests pass.
- A fresh exact-head `hermes-gpu-startup` fallback E2E passes on #9323 after the repair.
- Cleanup still destroys only transaction-owned resources and releases the gateway port.
Prerequisite for #9140 and PR #9323.
Contributor guide
Research direction
Start with the focused lifecycle and Hermes GPU fallback tests, then inspect the hermes-gpu-startup fallback E2E and the cleanup handoff result used by the retry decision. Done means deterministic coverage passes, the exact-head fallback E2E passes, and cleanup still destroys only transaction-owned resources while releasing the gateway port.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, typescript
- Domain
- backend, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 52/100