perf(cpu): add packed Cortex-A76 Q8 multi-output kernel
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Track KERNEL-CPU-A76-Q8-DOT follow-up work to close the measured Raspberry Pi 5 gap against llama.cpp by increasing data reuse rather than rescheduling the existing nrc=1 leaf.
Scope:
- profile and speed-limit model the current A76 Q8_0 x Q8_0 path;
- add one explicit, non-default packed 1x4 or 4x4 candidate through the shared MatmulBTQuant dispatch;
- preserve strict block-order floating-point accumulation and exact output/token gates;
- prove selector, packing/layout, tails, alignment, ABI and reporting with RED-first tests and mutations;
- require pinned GCC13 AArch64 disassembly plus QEMU correctness before any Pi measurement;
- compare same-binary current A76 assembly vs candidate on M=1 and reached prefill shapes with wall time, cycles, instructions, frontend/backend stalls and cache traffic;
- do not change the automatic default unless physical Pi and recursive model gates pass every throughput/latency/memory axis.
The current one-output assembly measures about 14.94 cycles per two Q8 blocks versus an optimistic ~10-cycle SIMD-port bound, while the full prefill profile attributes ~30% of cycles to Q8 and shows backend/cache rather than frontend pressure. The candidate hypothesis is activation reuse across four output rows, following the project-pinned llama.cpp 1x4/4x4 structure but adapting it to this project strict reduction-order contract.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the shared MatmulBTQuant dispatch and profile the current A76 Q8_0 x Q8_0 path on the pinned GCC13 AArch64 build. Add the non-default packed 1x4 or 4x4 candidate, then run RED-first selector, packing/layout, tail, alignment, ABI, mutation, and QEMU correctness checks. Done requires pinned disassembly and same-binary M=1/reached-prefill comparisons; do not change defaults until Pi and recursive model gates pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, raspberry-pi
- Domain
- backend, performance, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100