Flaky test: L1_Functional_Tests_Megatron_4 fails with no error, runner appears to die mid-test (14x in 11 days)

## Summary

`L1_Functional_Tests_Megatron_4` (mcore/Megatron generation functional test job, `fast_` variant included) has failed intermittently 14 times over the last ~11 days (2026-09-04 to 2026-09-15) with **no application-level error at all** — no Python traceback, no assertion/convergence failure, nothing in the test's own logs explaining the failure. The job simply stops abruptly after ~3m50s–4m20s (a full run normally takes 10–20+ min), Ray's own processes (gcs_server, raylet, etc.) get killed, the wrapper script prints `Did not finish.`, and the job fails with generic `Process completed with exit code 1.`

## Common signature (identical across all 14 occurrences)

```
Did not finish.
##[error]❌ L1_Functional_Tests_Megatron_4 — FAILED
##[error]Process completed with exit code 1.
```

- Job runtime: ~3m50s–4m20s (vs. normal 10–20+ min for this job)
- No Python traceback / no exception
- No convergence/assertion table failure
- Ray processes (`gcs_server`, raylet, etc.) shut down via SIGKILL shortly before the job ends
- Followed by a `Cleaning up orphan processes` log line
- Happens across PR pushes, `main` pushes, and the nightly `schedule` run — not tied to any specific PR/branch

One originating workflow run (33906565813) shows an overall run `conclusion` of `startup_failure` rather than `failure`.

## Occurrences (14, 2026-09-04 → 2026-09-15)

| Date (UTC) | Job | Run | Branch/PR |
|---|---|---|---|
| 2026-09-04 21:56 | [L1_Functional_Tests_Megatron_4](https://github.com/NVIDIA-NeMo/RL/actions/runs/33906565813/job/101184017833) | [33906565813](https://github.com/NVIDIA-NeMo/RL/actions/runs/33906565813) | PR #3991 |
| 2026-09-06 18:52 | [L1_Functional_Tests_Megatron_4](https://github.com/NVIDIA-NeMo/RL/actions/runs/34047641496/job/101540042869) | [34047641496](https://github.com/NVIDIA-NeMo/RL/actions/runs/34047641496) | PR #4028 |
| 2026-09-07 09:21 | [L1_Functional_Tests_Megatron_4](https://github.com/NVIDIA-NeMo/RL/actions/runs/34093342031/job/101689581581) | [34093342031](https://github.com/NVIDIA-NeMo/RL/actions/runs/34093342031) | PR #4009 |
| 2026-09-07 10:19 | [L1_Functional_Tests_Megatron_4](https://github.com/NVIDIA-NeMo/RL/actions/runs/34094016520/job/101706373513) | [34094016520](https://github.com/NVIDIA-NeMo/RL/actions/runs/34094016520) | PR #4040 |
| 2026-09-07 19:16 | [L1_Functional_Tests_Megatron_4](https://github.com/NVIDIA-NeMo/RL/actions/runs/34146500943/job/101844375772) | [34146500943](https://github.com/NVIDIA-NeMo/RL/actions/runs/34146500943) | PR #4043 |
| 2026-09-09 09:42 | [L1_Functional_Tests_Megatron_4](https://github.com/NVIDIA-NeMo/RL/actions/runs/34320496994/job/102414068800) | [34320496994](https://github.com/NVIDIA-NeMo/RL/actions/runs/34320496994) | PR #3613 |
| 2026-09-09 12:09 | [L1_Functional_Tests_Megatron_4](https://github.com/NVIDIA-NeMo/RL/actions/runs/34332811853/job/102458752365) | [34332811853](https://github.com/NVIDIA-NeMo/RL/actions/runs/34332811853) | `main` (nightly schedule) |
| 2026-09-10 01:36 | [L1_Functional_Tests_Megatron_4](https://github.com/NVIDIA-NeMo/RL/actions/runs/34416463438/job/102711642497) | [34416463438](https://github.com/NVIDIA-NeMo/RL/actions/runs/34416463438) | `main` (push) |
| 2026-09-12 09:20 | [L1_Functional_Tests_Megatron_4](https://github.com/NVIDIA-NeMo/RL/actions/runs/34677441291/job/103531801125) | [34677441291](https://github.com/NVIDIA-NeMo/RL/actions/runs/34677441291) | `main` (push) |
| 2026-09-12 09:38 | [L1_Functional_Tests_Megatron_4](https://github.com/NVIDIA-NeMo/RL/actions/runs/34658834843/job/103533531332) | [34658834843](https://github.com/NVIDIA-NeMo/RL/actions/runs/34658834843) | PR #4099 |
| 2026-09-14 05:49 | [L1_Functional_Tests_Megatron_4](https://github.com/NVIDIA-NeMo/RL/actions/runs/34805917217/job/103872189613) | [34805917217](https://github.com/NVIDIA-NeMo/RL/actions/runs/34805917217) | `workflow_dispatch` on `mcore-testing-6933` |
| 2026-09-14 11:11 | [L1_Functional_Tests_Megatron_4](https://github.com/NVIDIA-NeMo/RL/actions/runs/34826294634/job/103952709670) | [34826294634](https://github.com/NVIDIA-NeMo/RL/actions/runs/34826294634) | `main` (nightly schedule) |
| 2026-09-14 19:59 | [L1_Functional_Tests_Megatron_4](https://github.com/NVIDIA-NeMo/RL/actions/runs/34876537278/job/104130491855) | [34876537278](https://github.com/NVIDIA-NeMo/RL/actions/runs/34876537278) | PR #4113 |
| 2026-09-15 05:08 | [L1_Functional_Tests_Megatron_4](https://github.com/NVIDIA-NeMo/RL/actions/runs/34918826302/job/104260485073) | [34918826302](https://github.com/NVIDIA-NeMo/RL/actions/runs/34918826302) | PR #3613 |

## Notes

- All 14 are the `L1_Functional_Tests_Megatron_4` job family (runs `grpo_megatron_generation_colocated.sh`, `grpo_megatron_generation_non_colocated.sh`, `grpo_megatron_nccl_reshard_refit.sh`, etc. — the mcore/Megatron generation backend functional tests in `.github/workflows/cicd-main.yml`).