ROCm: release captured graphs when instantiation fails
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: BACKEND-ROCM
At e1948ecebde085907171abce41a7b2e7541fd7d8, RocmBackend::EndCaptureGraph obtains a raw hipGraph_t from hipStreamEndCapture, then checks hipGraphInstantiate before destroying that raw graph (src/vt/rocm/rocm_backend.hip:360–377). An instantiate error throws past hipGraphDestroy, so the captured raw graph loses its cleanup path. The single-slot EndCapture path has the same ordering at lines 342–351.
This is a source-confirmed cleanup gap. Controlled hardware error-injection evidence is still pending. The graph seam originated in 7ba9a675f (#473). The open-issue search for hipGraphInstantiate returned #332, whose broader historical activation issue does not track this failure cleanup.
Fix this in the current #3041 graph flow under its committed .agents/specs/rocm-gfx1100-static-graph.md capture-failure and partial-construction obligations. Preserve the primary instantiate error, destroy owned raw handles exactly once, and retain successful capture/replay behavior. Cover both affected entry points and any partial executable ownership without changing provider, dtype, sampling, graph-default or dedup policy.
Before the repair, run a small immutable test against the unchanged baseline. It must enter the real backend, capture a real HIP graph, inject one instantiate failure, and observe raw-handle cleanup. Preserve the intended red, focused green, effective deletion mutation, fresh review and operator rerun. Root holds the local GPU mutex. The landing PR must reference this issue and close it when the repair lands; filing it does not defer the fix.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in src/vt/rocm/rocm_backend.hip at RocmBackend::EndCaptureGraph and the single-slot EndCapture path, then read the committed .agents/specs/rocm-gfx1100-static-graph.md obligations. Run the immutable baseline test against a real HIP graph with one injected instantiate failure. Done means both entry points preserve the primary error, release owned raw handles exactly once, and retain successful capture/replay behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100