abetlen / abetlen/llama-cpp-python
Support for arm64 wheels and CPU Features
- Vorherrschende Sprache
- Python
- Sterne
- 10.6k
- Forks
- 1.4k
- PR-Merge-Kennzahlen
- PR-Kennzahlen ausstehend
Beschreibung
@abetlen Thank you for the new efforts to start publishing wheels for CUDA, etc.
I noticed that the METAL wheels only work for darwin platform, when using Docker in MacOS the platform is arm64/linux not darwin.
I have a repo where I was building arm64/wheels that could probably be integrated into your workflows: https://github.com/gaby/arm64-wheels
TLDR
```yaml
steps:
- name: Checkout abetlen/llama-cpp-python
uses: actions/checkout@v4
with:
repository: 'abetlen/llama-cpp-python'
ref: '${{ matrix.version }}'
submodules: 'recursive'
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
with:
platforms: linux/arm64
- name: Build wheels
uses: pypa/cibuildwheel@v2.16.5
env:
CIBW_SKIP: "*musllinux* pp*"
CIBW_REPAIR_WHEEL_COMMAND: ""
CIBW_ARCHS: "aarch64"
CIBW_BUILD: "cp311-*"
with:
output-dir: wheelhouse/
- name: Upload wheels as artifacts
uses: actions/upload-artifact@v4
with:
name: wheels-${{ matrix.version }}
path: wheelhouse/*.whl
```
This would need to be expanded to support other Python versions/Pypy.
I also notice the CPU wheels don't have specifics about AVX, AVX2, AVX512 are there plans to add support for those?
Beitragsleitfaden
Bewertung
Dieses Issue wurde noch nicht bewertet.