[BUG]: VMM growth frees an unowned address after a failed adjacent reservation

未关闭
#2,345 0 条评论 0 个 reaction 已指派 1 人 在 GitHub 查看

@Andy-Jost 已经在做这个了。

开始于 2026年7月23日。

评估

这个 Issue 还没有评估数据。

描述

Component

cuda.core

What happened?

VirtualMemoryResource.modify_allocation() first tries to reserve the additional virtual address range immediately after the existing allocation.

When that cuMemAddressReserve() call fails with a recoverable error, the returned pointer is not an owned reservation. The cuda.bindings wrapper returns (error, None) for this case, but the current growth path still calls cuMemAddressFree(new_ptr, size) before entering the slow-path fallback.

That cleanup call is invalid because no reservation was created. It can replace the original recoverable reservation error with another driver error, such as CUDA_ERROR_INVALID_VALUE, and prevent the fallback from being attempted.

How to reproduce

On a CUDA VMM-capable GPU, first allocate a small buffer and request an intentionally impossible growth size:

import platform

from cuda.core import Device, VirtualMemoryResource, VirtualMemoryResourceOptions
from cuda.core._utils.cuda_utils import CUDAError


device = Device(0)
device.set_current()

handle_type = "win32_kmt" if platform.system() == "Windows" else "posix_fd"
mr = VirtualMemoryResource(
    device,
    config=VirtualMemoryResourceOptions(handle_type=handle_type),
)

buf = mr.allocate(4096)
try:
    # Adjust this value if needed to make cuMemAddressReserve return
    # CUDA_ERROR_OUT_OF_MEMORY on the target platform.
    mr.modify_allocation(buf, 1 << 62)
except CUDAError as exc:
    print(exc)
finally:
    buf.close()

On the affected path, when the adjacent reservation returns CUDA_ERROR_OUT_OF_MEMORY, the subsequent cuMemAddressFree(None, size) produces a second error and masks the original reservation failure. The corrected path skips the free because the reservation did not succeed, then proceeds to the slow-path attempt and preserves the driver error if that attempt also fails.

The binding-level behavior can also be observed directly: cuMemAddressReserve() returns ptr=None on failure, while cuMemAddressFree() requires a pointer that came from a successful reservation.

Expected behavior

If the adjacent reservation fails, the code should not call cuMemAddressFree() for its returned pointer. It should either enter the slow path or propagate the original reservation error.

A reservation should only be freed when cuMemAddressReserve() succeeded and returned a noncontiguous address.

Additional context

PR #2237 separates these two cases and adds regression coverage for both failed reservations and successful noncontiguous reservations.

主要语言
Cython
星标
3.4k
派生
329
平均合并
1 天 21 小时
30 天内合并 PR
113

贡献指南

打开贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

NVIDIA/cuda-python 的其他 Issue

查看 NVIDIA/cuda-python 的全部 Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。