abetlen / abetlen/llama-cpp-python

Support for arm64 wheels and CPU Features

Offen
#1,342 11 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
Python
Sterne
10.6k
Forks
1.4k
PR-Merge-Kennzahlen
PR-Kennzahlen ausstehend

Beschreibung

@abetlen Thank you for the new efforts to start publishing wheels for CUDA, etc.

I noticed that the METAL wheels only work for darwin platform, when using Docker in MacOS the platform is arm64/linux not darwin.

I have a repo where I was building arm64/wheels that could probably be integrated into your workflows: https://github.com/gaby/arm64-wheels

TLDR
```yaml
steps:
- name: Checkout abetlen/llama-cpp-python
uses: actions/checkout@v4
with:
repository: 'abetlen/llama-cpp-python'
ref: '${{ matrix.version }}'
submodules: 'recursive'

- name: Set up QEMU
uses: docker/setup-qemu-action@v3
with:
platforms: linux/arm64

- name: Build wheels
uses: pypa/cibuildwheel@v2.16.5
env:
CIBW_SKIP: "*musllinux* pp*"
CIBW_REPAIR_WHEEL_COMMAND: ""
CIBW_ARCHS: "aarch64"
CIBW_BUILD: "cp311-*"
with:
output-dir: wheelhouse/

- name: Upload wheels as artifacts
uses: actions/upload-artifact@v4
with:
name: wheels-${{ matrix.version }}
path: wheelhouse/*.whl
```

This would need to be expanded to support other Python versions/Pypy.

I also notice the CPU wheels don't have specifics about AVX, AVX2, AVX512 are there plans to add support for those?

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.