NVIDIA / NVIDIA/cuda-python

Lowering `StridedMemoryView` attribues to typed efficient C/C++/Cython accessible values

Open
#180 2 comments 1 reaction 1 assignee View on GitHub

@leofang is already working on this.

Since Oct 17, 2024.

cuda.core enhancement P1
Dominant language
Cython
Stars
3.4k
Forks
329
Avg merge
1d 23h
Merged PRs (30d)
116

Description

Currently the attributes of StridedMemoryView are as follows:

https://github.com/NVIDIA/cuda-python/blob/8c841cdb24f64e65138cd2658d30fdeabd18769b/cuda_core/cuda/core/experimental/_memoryview.pyx#L24-L32

There is a todo in the code noting that this is worth converting to Cython types. Would also support this recommendation

When accessing things like shape or strides in Cython, one ideally wants a C array type of some form (pointer, typed-memoryview, dynamic array, etc.) that can easily be iterated over in C for-loops. As these attributes are currently accessing them requires calling the CPython API to get the length, each value, coerce them to C friendly types, etc.

This is especially important for things like ptr, which gets accessed regularly. So having a fast access C-type really helps


If we look at the Python Buffer Protocol (PEP 3118), they have the following definition for Py_buffer (their equivalent type):

typedef struct {
    void *buf;
    PyObject *obj;        /* owned reference */
    Py_ssize_t len;
    Py_ssize_t itemsize;  /* This is Py_ssize_t so it can be
                             pointed to by strides in simple case.*/
    int readonly;
    int ndim;
    char *format;
    Py_ssize_t *shape;
    Py_ssize_t *strides;
    Py_ssize_t *suboffsets;
    void *internal;
} Py_buffer;

They also require users to call PyObject_GetBuffer to produce a buffer object and PyBuffer_Release to release a buffer object. This handles any memory allocation/deallocation for shape, strides, etc.. It also handles refcounting for obj. This functionality is wonderful to use

The lack of these semantics has made working with DLPack a chore


Thinking about how to map the C-like struct above to Cython/Python. A few things stick out

Some of these Cython can translate between Python/Cython/C like Py_ssize_t (effectively ssize_t in C (PEP 353)) to Python int's when needed.

Others can be coerced like char* to bytes (though usually one will want to decode/encode to/from str)

Still others could be translated well by Cython as long as they are typed appropriately. For example void* doesn't translate well to Python. However uintptr_t does behave like a pointer in C (sometimes with a cast) and like a Python int. Cython will handle the translation for us. Similarly bint for readonly works better when capturing Python's bool semantics while still working in C.

With a typed memoryview, it is possible to wrangle Py_ssize_t* into something better behaved like Py_ssize_t[::1], which can then move more easily between Python & Cython.

Also note that format above is a string specifying the format type according to the Python Buffer Protocol. NumPy is also able to consume and produce such format strings

This led RAPIDS to this approach:

cdef class Array:
    cdef readonly uintptr_t ptr
    cdef readonly bint readonly
    cdef readonly object obj

    cdef readonly Py_ssize_t itemsize

    cdef readonly Py_ssize_t ndim
    cdef Py_ssize_t[::1] shape_mv
    cdef Py_ssize_t[::1] strides_mv

    cdef readonly bint cuda

If StridedMemoryView is used with C/C++ directly, it may make sense to actually type a public C struct in Cython. Then this could be leveraged in C/C++ code that can hand such objects to or receive them from CUDA-Python. For example

# filename: strided_memory_view.pxd

from libc.stdint cimport uintptr_t

cdef public struct CStridedMemoryView:
    uintptr_t ptr
    ssize_t* shape
    ssize_t* strides
    char* format
    int device_id
    bint device_accessible
    bint readonly
    void* obj

cdef class StridedMemoryView:
    CStridedMemoryView data
// filename: my_program.c

#include <stdbool.h>
#include <stdint.h>

#include <sys/types.h>

#include "strided_memory_view.h"


static const char* s = "Hello World!";
static const ssize_t s_len = 13;


int main() {
    CStridedMemoryView st;
    st.ptr = (uintptr_t)s;
    st.shape = &s_len;
    st.readonly = true;

    // ...

    return 0;
}

Though there can be other valid ways to go

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.