e2e(gateway): guard-chain recovery intermittently health-times out after recreation
- Dominant language
- TypeScript
- Stars
- 22.5k
- Forks
- 3.1k
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 715
Description
## Investigation Summary
- [Gateway guard-chain recovery](https://github.com/NVIDIA/NemoClaw/actions/runs/33730208977/job/100568886794) fails on `3076188279c0ea657ec77aeeabebf1f82061eaaf` with `GATEWAY_HEALTH_TIMEOUT` after legacy recovery/recreation.
- Failure evidence includes invalid/stale PID or process-stage observations before the health deadline.
- The [same target passes](https://github.com/NVIDIA/NemoClaw/actions/runs/33761515943/job/100678908962) on later main candidate `eb3fa2e5097389d7dd8e2643552788ff16d8f14a`.
- Closed issue #7484 documents the broader supervised-gateway timeout family; this tracker covers recurrence in the guard-chain recreation E2E.
## Description
Gateway guard recovery is timing-sensitive after recreation. The outer controller can time out without identifying whether the replacement process, PID handoff, listener, or health surface lagged.
Expected: the replacement gateway reaches a stable health state, with actionable stage-specific evidence if it does not.
Actual: the E2E receives a generic health timeout and passes on rerun.
## Reproduction Steps
1. Run the guard-chain recreation Docker E2E repeatedly under CI load.
2. Allow the target to recreate and recover the legacy gateway.
3. Observe intermittent PID/process transition failure followed by `GATEWAY_HEALTH_TIMEOUT`.
## Environment
- GitHub Actions Docker E2E
- Failed run: [33730208977](https://github.com/NVIDIA/NemoClaw/actions/runs/33730208977)
- Passing comparison: [33761515943](https://github.com/NVIDIA/NemoClaw/actions/runs/33761515943)
## Acceptance Criteria
- Record supervisor PID, child PID, listener readiness, and health stage at failure.
- Make replacement PID handoff robust against expected startup latency.
- Demonstrate repeated guard-chain recovery passes under CI load.
Contributor guide
Assessment
This issue has not been assessed yet.