abetlen / abetlen/llama-cpp-python
Support for arm64 wheels and CPU Features
- 主要语言
- Python
- 星标
- 10.6k
- 派生
- 1.4k
- PR 合并指标
- PR 指标待抓取
描述
@abetlen Thank you for the new efforts to start publishing wheels for CUDA, etc.
I noticed that the METAL wheels only work for darwin platform, when using Docker in MacOS the platform is arm64/linux not darwin.
I have a repo where I was building arm64/wheels that could probably be integrated into your workflows: https://github.com/gaby/arm64-wheels
TLDR
```yaml
steps:
- name: Checkout abetlen/llama-cpp-python
uses: actions/checkout@v4
with:
repository: 'abetlen/llama-cpp-python'
ref: '${{ matrix.version }}'
submodules: 'recursive'
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
with:
platforms: linux/arm64
- name: Build wheels
uses: pypa/cibuildwheel@v2.16.5
env:
CIBW_SKIP: "*musllinux* pp*"
CIBW_REPAIR_WHEEL_COMMAND: ""
CIBW_ARCHS: "aarch64"
CIBW_BUILD: "cp311-*"
with:
output-dir: wheelhouse/
- name: Upload wheels as artifacts
uses: actions/upload-artifact@v4
with:
name: wheels-${{ matrix.version }}
path: wheelhouse/*.whl
```
This would need to be expanded to support other Python versions/Pypy.
I also notice the CPU wheels don't have specifics about AVX, AVX2, AVX512 are there plans to add support for those?
贡献指南
评估
这个 Issue 还没有评估数据。