NVIDIA / NVIDIA/cuda-python

[FEA] cuda.core: support logical endpoints (CUDA 13.3 unicast/multicast memory sharing)

Open
#2,358 0 comments 0 reactions 1 assignee View on GitHub

@juenglin is already working on this.

Since Jul 24, 2026.

cuda.core feature
Dominant language
Cython
Stars
3.4k
Forks
329
Avg merge
1d 23h
Merged PRs (30d)
116

Description

Summary

CUDA 13.3 introduces logical endpoints
(driver docs):
ID-addressed endpoints that memory allocations are bound to, in two flavors:

  • unicast — single-device endpoint; a designated owner device manages the binding
  • multicast — a team of devices shares the bound allocation

Endpoints are created on reserved ID ranges, constructed asynchronously (completion polled via
cuLogicalEndpointQuery), and shareable across processes via fabric IPC handles. A
CU_LOGICAL_ENDPOINT_FLAG_COUNTED_OPS flag declares intent to use counted operations. Four new
device attributes advertise support. cuda.bindings 13.3.x exposes all of it; cuda.core uses
none of it.

Structurally this generalizes the cuMulticast* flow (add-device / bind-mem / bind-addr /
unbind) that #2057 proposes to wrap. The driver docs do not state a formal relationship between
the two API families, but the two designs should be settled together — e.g. build
MulticastObject on logical endpoints when available (driver ≥ 13.3), with classic
cuMulticast* as fallback.

Underlying C APIs to cover

C API Purpose
cuLogicalEndpointIdReserve(CUlogicalEndpointId* baseLeId, cuuint32_t count) / cuLogicalEndpointIdRelease(baseLeId, count) reserve / release a contiguous range of endpoint IDs
cuLogicalEndpointCreate(CUlogicalEndpointId leId, const CUlogicalEndpointProp* prop) / cuLogicalEndpointDestroy(leId) create / destroy an endpoint on a reserved ID
cuLogicalEndpointAddDevice(leId, CUdevice dev) enroll a device in a multicast endpoint's team
cuLogicalEndpointBindMem(leId, dev, offset, CUmemGenericAllocationHandle memHandle, memOffset, size, flags) bind a VMM allocation at an endpoint offset
cuLogicalEndpointBindAddr(leId, dev, offset, void* ptr, size, flags) bind virtually-addressed memory
cuLogicalEndpointUnbind(leId, dev, offset, size) unbind a range
cuLogicalEndpointExport(void* handle, leId, CUlogicalEndpointIpcHandleType) / cuLogicalEndpointImport(leId, handle, handleType) IPC export / import (..._IPC_HANDLE_TYPE_FABRIC)
cuLogicalEndpointQuery(leId, cuuint32_t count, int* queryStatus) poll construction completion over an ID range
cuLogicalEndpointGetLimits(cuuint64_t* bindAlignment, cuuint64_t* maxSize, const CUlogicalEndpointProp*) alignment / max-size constraints for given properties

Supporting types: CUlogicalEndpointId, CUlogicalEndpointProp (type, unicast.device /
multicast.numDevices, size, ipcHandleTypes, flags), CUlogicalEndpointFabricHandle;
enums CUlogicalEndpointType, CUlogicalEndpointFlag, CUlogicalEndpointIpcHandleType.

New device attributes (also listed in the separate device-attributes catch-up item):
CU_DEVICE_ATTRIBUTE_LOGICAL_ENDPOINT_{UNICAST,MULTICAST,COUNTED_OPS,UNICAST_ACCESS_ON_OWNER_DEVICE}_SUPPORTED.

Design sketch (draft — needs design-meeting review)

[!IMPORTANT]
Starting point only, not a settled design — to be reviewed in the cuda.core design
meeting, together with #2057.

ep = LogicalEndpoint(                       # all names TBD; reserves an ID + creates
    size=nbytes,
    kind="multicast",                       # or "unicast"
    devices=[dev0, dev1],                   # multicast team (unicast: the owner device)
    counted_ops=False,
)
ep.bind(vmm_buffer, endpoint_offset=0)      # BindMem vs BindAddr chosen from input type
handle = ep.export()                        # fabric handle (bytes-like)
peer_ep = LogicalEndpoint.from_handle(handle)   # import in another process

Open questions for the meeting:

  1. ID-range reservation: hide it (one ID per object) or expose batch reserve+create for
    scale-out patterns (cuLogicalEndpointQuery polls a whole range at once)?
  2. Async construction: block in the constructor, or expose an is_ready-style waiter?
  3. bind() inputs: Buffer backed by VirtualMemoryResource (owns the
    CUmemGenericAllocationHandle) vs. raw pointers — support both?
  4. Relationship with #2057: one abstraction with two backends, or two classes?
  5. IPC surface: align with existing cuda.core IPC patterns (mempool/event handles).

References

-- Leo's bot

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.