abetlen / abetlen/ggml-python

Investigate different approaches to binding to ggml

未關閉
#3 0 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
主要語言
Python
星號
152
分支
16
PR 合併指標
30 天內沒有已合併 PR

描述

**Current Approach**
Use scikit-build-core and a CMakeLists file in the root directory to compile ggml as a shared library and install it relative to the `ggml/ggml.py` file so that it can be loaded by `ctypes.CDLL`.

**Issues with Current Approach**
- `#define`'s are not accessible from the shared library, this is an issue for values that are computed on the host system at compile time.
- Platform specific functions such as those for CUDA, OpenCL, and Metal are also awkward to conditionally support with the ctypes approach
- Assuming the relative location of files in an installed package causes issues, especially when installing in ie. editable mode for local development.
- Some arguments such as ctypes Array's require awkward typing workarounds.

**Alternative Binding Systems**
- Cython
- Pybind11
- [Nanobind (WIP)](https://github.com/abetlen/ggml-python/tree/test-nanobind-build-system)
- [SWIG (WIP)](https://github.com/abetlen/ggml-python/tree/test-swig-build-system)
- CFFI(?)
- Stick with ctypes

**Requirements**
- Should be at least as fast as ctypes approach (large overhead is not acceptable for this application)
- Should be simple to maintain (ggml changes often, should be at least as simple as ctypes to add new function definitions / types / etc)
- Should be easy for others to extend or modify ggml
- Should use ggml's existing build system and flags when compiling, should not limit which platforms / optimizations can be used

貢獻指南

開啟貢獻指南

評估

這個 Issue 還沒有評估資料。

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。