NVIDIA / NVIDIA/cuda-python

cuda.core: return CUresult from C++ handle factories instead of thread-local error state

Open
#2,760 1 comment 0 reactions 1 assignee View on GitHub

@Andy-Jost is already working on this.

Since Sep 14, 2026.

cuda.core enhancement P1
Dominant language
Cython
Stars
3.4k
Forks
329
Avg merge
1d 23h
Merged PRs (30d)
116

Description

Summary

The C++ handle layer (cuda_core/cuda/core/_cpp/resource_handles.*) uses two status conventions. Factories return the handle and stash the CUresult in thread-local err, read back with get_last_error(). Functions that do not produce a handle (context_synchronize, context_get_device, graph_node_set_params, the graph_*_attachment family, deviceptr_alloc_raw, ...) return the CUresult directly with results in out-parameters, like the driver API. Proposal: use the second convention everywhere and remove the thread-local error state.

Why

The documented justification for thread-local err is that factories can be called from nogil code without acquiring the GIL on the success path. HANDLE_RETURN is itself nogil and only takes the GIL on a non-success status, so a CUresult-returning factory has the same property. What is left is expression-style ergonomics (h = create_stream_handle(...)), which would only pay off with a real exception channel, and this layer cannot have one.

Unifying on direct CUresult returns would:

  • Remove the read-before-clobber footgun and the get_last_error / peek_last_error / clear_last_error trio.
  • Remove an ambiguity that exists today: an empty handle can mean a legitimate "none" (get_current_context() with no current context, get_context_green_ctx() on a plain context) or a failure, and only err tells them apart. With CUresult f(Handle* out, ...), success plus an empty out-handle is unambiguous.
  • Make internal composition explicit propagation instead of "the callee already set err, return an empty handle".
  • Give secondary statuses a natural home as out-parameters (graph_node_set_params already does this with restore_status).

Proposed rule

Anything that can fail returns CUresult and delivers its result through an out-parameter, mirroring the driver. Accessors that cannot fail (get_stream_context, get_event_device_id, the *_ref constructors, ...) keep returning values directly. One rule, no exceptions keyed on return type.

Cython call sites become:

cdef StreamHandle h
HANDLE_RETURN(create_stream_handle(&h, h_ctx, flags, priority))

Scope

Rough counts on the current tree: 66 handle-returning factory signatures, 37 internal err = assignments in the C++ layer, and 21 Cython sites in 13 modules that read err, plus every Cython factory call. Mechanical and behavior-neutral, but it touches every module, so it should land as a standalone change with no functional edits mixed in, after #2750 and #2759 merge. It is independent of the planned follow-up on calling raw driver function pointers instead of the Cython wrappers.

Non-goals

  • No behavior change; every status that is raised or reported today continues to be raised or reported.
  • The thread-local last_error_detail buffer added by #2759 is ancillary message text consumed only at the raise point, not control-flow state. It can stay as is; folding it into a status struct is a possible later step.

Refs: #2758 (error-handling policy RFC), #2759 (implementation), review discussion on _cpp/DESIGN.md in #2759.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.