cuda.core: VirtualMemoryResource grow path's Buffer._clear() re-triggers mr.deallocate() on an already-freed VA range
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 52/100
Research direction
Start with _grow_allocation_slow_path in cuda_core/cuda/core/_memory/_virtual_memory_resource.py and Buffer._clear() in _buffer.pyx. Run test_vmm_allocator_grow_allocation and inspect the Buffer.from_handle() and deallocate() paths to verify whether clearing the old buffer invokes its deleter. Done means the grow path no longer attempts to deallocate an already-freed VA range or emits the reported CUDAWarning.
Written by the indexing model from the issue text.
Description
Summary
_grow_allocation_slow_path (cuda_core/cuda/core/_memory/_virtual_memory_resource.py:362) frees the old buffer's VA range manually (cuMemAddressFree(int(buf.handle), aligned_prev_size), line 457), then calls buf._clear() (line 460) with a comment saying this stops the old buffer's destructor from freeing it again.
Buffer._clear() (_buffer.pyx:262) does self._h_ptr.reset(). For a shared_ptr, reset() isn't a no-op invalidation — if buf holds the last reference, it runs the deleter immediately. This buffer's handle was created with mr=self (Buffer.from_handle(..., mr=self) in allocate()), so its deleter calls back into mr.deallocate() with the old, already-freed pointer and size. deallocate() then calls cuMemRetainAllocationHandle on a VA range that no longer exists, which fails with CUDA_ERROR_INVALID_VALUE. Under the error-handling policy landed in #2759, that failure is caught in the deleter and reported as a CUDAWarning instead of being silent; previously it was silently swallowed.
Evidence
Seen in CI, test_vmm_allocator_grow_allocation:
_virtual_memory_resource.py:292: CUDAWarning: mr.deallocate() failed during Buffer
destruction; the allocation may have leaked: CUDA_ERROR_INVALID_VALUE: This indicates
that one or more of the parameters passed to the API call is not within an acceptable
range of values.
Hypothesis, not yet confirmed by reproduction
Read from the code during review of #2759, not verified by running the test with added instrumentation.
Suggested fix direction
_clear()'s intent here seems to be "drop bookkeeping for a range this code already freed by hand," not "release the handle normally." Those need to be different operations: either give Buffer a way to drop its handle without invoking the resource's deallocate() (e.g. release the underlying pointer from the shared_ptr without running the deleter), or have the grow path swap in a handle whose deleter is already a no-op.
Refs: found during review of #2759.
- Dominant language
- Cython
- Stars
- 3.4k
- Forks
- 329
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 113
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from NVIDIA/cuda-python
-
bug cuda.core
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
NVIDIA/cuda-python#2886 · 1 comment ·
-
triage
Difficulty 1/5 Under an hour Newbie friendliness 88/100
NVIDIA/cuda-python#2717 ·
-
triage
Difficulty 1/5 1-3 hours Newbie friendliness 90/100
NVIDIA/cuda-python#2712 ·
-
triage
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
NVIDIA/cuda-python#2646 · 1 reaction ·
-
cuda.core triage
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
NVIDIA/cuda-python#2435 · 1 comment ·
All issues in NVIDIA/cuda-python
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
code-quality refactoring
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
github/gh-aw-firewall#8816 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
getgrav/grav-plugin-api#45 ·
-
integration:quickjs org:external priority:backlog topic:code-interpreter topic:middleware type:feature
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
langchain-ai/deepagents#6450 ·