🐛 CI hang/timeout: hybrid_dynamic_inference_tp1_ep8_nanov3_chunked_prefill exceeds 1h job limit on repeat iteration 3

**Describe the bug**

CI job [`hybrid/hybrid_dynamic_inference_tp1_ep8_nanov3_chunked_prefill - latest`](https://github.com/NVIDIA/Megatron-LM/actions/runs/25416016966/job/74655244310) was canceled after exceeding the **1h0m0s** job timeout. The failing test node is `tests/functional_tests/python_test_utils/test_inference_regular_pipeline.py::test_inference_pipeline`, run with `REPEAT=3`.

The first two repeat iterations completed and PASSED (at ~17:00 and ~17:03, ~43 min into the job). The 3rd iteration hung after the inference coordinator finished startup. The last meaningful log line before the runner forcibly cancelled the job at 17:20:56 was:

```
INFO:root:Inference Coordinator: Connected with data parallel ranks...
INFO:root:Inference co-ordinator is ready to receive requests!
```

…then ~17 minutes of silence until `##[error]The operation was canceled.`. Note that the job-step framework's exit-code branch printed `✅ PASSED` (because the prior iterations succeeded and the cancellation arrived during teardown/3rd-iter execution rather than as a test assertion), but the GitHub job conclusion is `cancelled` and the run is treated as failed.

Tag the [@mcore-oncall](https://github.com/orgs/NVIDIA/teams/mcore-oncall) to get oncall's attention to this issue.

**Failing run**

| Field | Value |
|-------|-------|
| PR | [#4611: chore: Update Docker image version to 26.04-py3](https://github.com/NVIDIA/Megatron-LM/pull/4611) |
| Run | [25416016966](https://github.com/NVIDIA/Megatron-LM/actions/runs/25416016966) |
| Job | [hybrid/hybrid_dynamic_inference_tp1_ep8_nanov3_chunked_prefill - latest](https://github.com/NVIDIA/Megatron-LM/actions/runs/25416016966/job/74655244310) |
| Job timing | started 2026-05-06T16:20:27Z, completed 2026-05-06T17:20:59Z, conclusion `cancelled` |

**Error**

```
##[error]The operation was canceled.

# Last meaningful log lines before cancellation (repeat iter 3):
[default0]:LANG_RL Log directory: None
[default0]:/opt/venv/lib/python3.12/site-packages/mamba_ssm/utils/determinism.py:73: UserWarning: Deterministic mode: ...
[default0]:INFO:root:Inference Coordinator: waiting for connections from data parallel ranks...
[default0]:INFO:root:Inference Coordinator: Connected with data parallel ranks...
[default7]: ...
[default0]:INFO:root:Inference co-ordinator is ready to receive requests!
# (no further output for ~17 minutes)
##[error]The operation was canceled.
```

(Full log available at the job URL above. Iters 1 and 2 of the inference test PASSED at lines 4719/4726 and 8051/8058 of the log respectively.)

**Steps/Code to reproduce bug**

Re-run the failing CI job linked above, or locally inside the dev container with `REPEAT=3`:

```bash
pytest tests/functional_tests/python_test_utils/test_inference_regular_pipeline.py::test_inference_pipeline
```

against the `hybrid_dynamic_inference_tp1_ep8_nanov3_chunked_prefill` test case. The hang reproduces on the third repeat iteration after the coordinator reports "ready to receive requests".

**Suggested next steps**

- Investigate why the inference coordinator on repeat iter 3 never receives client connections — likely a port/state leak from iters 1–2 (stale unix socket, not-fully-released GPU memory, or coordinator address collision in `gpt_dynamic_inference_with_coordinator.py`).
- As a short-term mitigation, either:
- reduce `REPEAT` for this test, or
- add a per-iteration timeout inside the test driver so a hung iter fails fast instead of consuming the whole job's 1h budget.

**Additional context**

Triaged automatically via `/create-issue`. The `hybrid_dynamic_inference_tp1_ep8_nanov3_chunked_prefill/model_config.yaml` was last touched by `@sidsingh-nvidia` (commit `bfd45740`, "New allgathervdispatcher for inference and simplify old dispatcher (#4258)").