abetlen / abetlen/ggml-python

Investigate different approaches to binding to ggml

Abierto
#3 0 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Python
Estrellas
152
Forks
16
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

**Current Approach**
Use scikit-build-core and a CMakeLists file in the root directory to compile ggml as a shared library and install it relative to the `ggml/ggml.py` file so that it can be loaded by `ctypes.CDLL`.

**Issues with Current Approach**
- `#define`'s are not accessible from the shared library, this is an issue for values that are computed on the host system at compile time.
- Platform specific functions such as those for CUDA, OpenCL, and Metal are also awkward to conditionally support with the ctypes approach
- Assuming the relative location of files in an installed package causes issues, especially when installing in ie. editable mode for local development.
- Some arguments such as ctypes Array's require awkward typing workarounds.

**Alternative Binding Systems**
- Cython
- Pybind11
- [Nanobind (WIP)](https://github.com/abetlen/ggml-python/tree/test-nanobind-build-system)
- [SWIG (WIP)](https://github.com/abetlen/ggml-python/tree/test-swig-build-system)
- CFFI(?)
- Stick with ctypes

**Requirements**
- Should be at least as fast as ctypes approach (large overhead is not acceptable for this application)
- Should be simple to maintain (ggml changes often, should be at least as simple as ctypes to add new function definitions / types / etc)
- Should be easy for others to extend or modify ggml
- Should use ggml's existing build system and flags when compiling, should not limit which platforms / optimizations can be used

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.