[FEA] cuda.core: support logical endpoints (CUDA 13.3 unicast/multicast memory sharing)
@juenglin ci sta già lavorando.
Dal 24/7/2026.
- Lingua principale
- Cython
- Stelle
- 3.4k
- Fork
- 329
- Merge medio
- 1g 23h
- PR unite (30g)
- 116
Descrizione
Summary
CUDA 13.3 introduces logical endpoints
(driver docs):
ID-addressed endpoints that memory allocations are bound to, in two flavors:
- unicast — single-device endpoint; a designated owner device manages the binding
- multicast — a team of devices shares the bound allocation
Endpoints are created on reserved ID ranges, constructed asynchronously (completion polled via
cuLogicalEndpointQuery), and shareable across processes via fabric IPC handles. A
CU_LOGICAL_ENDPOINT_FLAG_COUNTED_OPS flag declares intent to use counted operations. Four new
device attributes advertise support. cuda.bindings 13.3.x exposes all of it; cuda.core uses
none of it.
Structurally this generalizes the cuMulticast* flow (add-device / bind-mem / bind-addr /
unbind) that #2057 proposes to wrap. The driver docs do not state a formal relationship between
the two API families, but the two designs should be settled together — e.g. build
MulticastObject on logical endpoints when available (driver ≥ 13.3), with classic
cuMulticast* as fallback.
Underlying C APIs to cover
| C API | Purpose |
|---|---|
cuLogicalEndpointIdReserve(CUlogicalEndpointId* baseLeId, cuuint32_t count) / cuLogicalEndpointIdRelease(baseLeId, count) |
reserve / release a contiguous range of endpoint IDs |
cuLogicalEndpointCreate(CUlogicalEndpointId leId, const CUlogicalEndpointProp* prop) / cuLogicalEndpointDestroy(leId) |
create / destroy an endpoint on a reserved ID |
cuLogicalEndpointAddDevice(leId, CUdevice dev) |
enroll a device in a multicast endpoint's team |
cuLogicalEndpointBindMem(leId, dev, offset, CUmemGenericAllocationHandle memHandle, memOffset, size, flags) |
bind a VMM allocation at an endpoint offset |
cuLogicalEndpointBindAddr(leId, dev, offset, void* ptr, size, flags) |
bind virtually-addressed memory |
cuLogicalEndpointUnbind(leId, dev, offset, size) |
unbind a range |
cuLogicalEndpointExport(void* handle, leId, CUlogicalEndpointIpcHandleType) / cuLogicalEndpointImport(leId, handle, handleType) |
IPC export / import (..._IPC_HANDLE_TYPE_FABRIC) |
cuLogicalEndpointQuery(leId, cuuint32_t count, int* queryStatus) |
poll construction completion over an ID range |
cuLogicalEndpointGetLimits(cuuint64_t* bindAlignment, cuuint64_t* maxSize, const CUlogicalEndpointProp*) |
alignment / max-size constraints for given properties |
Supporting types: CUlogicalEndpointId, CUlogicalEndpointProp (type, unicast.device /
multicast.numDevices, size, ipcHandleTypes, flags), CUlogicalEndpointFabricHandle;
enums CUlogicalEndpointType, CUlogicalEndpointFlag, CUlogicalEndpointIpcHandleType.
New device attributes (also listed in the separate device-attributes catch-up item):
CU_DEVICE_ATTRIBUTE_LOGICAL_ENDPOINT_{UNICAST,MULTICAST,COUNTED_OPS,UNICAST_ACCESS_ON_OWNER_DEVICE}_SUPPORTED.
Design sketch (draft — needs design-meeting review)
[!IMPORTANT]
Starting point only, not a settled design — to be reviewed in the cuda.core design
meeting, together with #2057.
ep = LogicalEndpoint( # all names TBD; reserves an ID + creates
size=nbytes,
kind="multicast", # or "unicast"
devices=[dev0, dev1], # multicast team (unicast: the owner device)
counted_ops=False,
)
ep.bind(vmm_buffer, endpoint_offset=0) # BindMem vs BindAddr chosen from input type
handle = ep.export() # fabric handle (bytes-like)
peer_ep = LogicalEndpoint.from_handle(handle) # import in another process
Open questions for the meeting:
- ID-range reservation: hide it (one ID per object) or expose batch reserve+create for
scale-out patterns (cuLogicalEndpointQuerypolls a whole range at once)? - Async construction: block in the constructor, or expose an
is_ready-style waiter? bind()inputs:Bufferbacked byVirtualMemoryResource(owns the
CUmemGenericAllocationHandle) vs. raw pointers — support both?- Relationship with #2057: one abstraction with two backends, or two classes?
- IPC surface: align with existing cuda.core IPC patterns (mempool/event handles).
References
- Driver docs: https://docs.nvidia.com/cuda/cuda-driver-api/group__CUDA__LOGICAL__ENDPOINT.html
- Found during the CUDA 12.8 → 13.3 bindings vs. cuda.core gap sweep (2026-07-14)
-- Leo's bot
Guida per i contributori
Apri la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Valutazione
Questa issue non è ancora stata valutata.