cactus-compute / cactus-compute/needle
[Discussion] Pure-Rust + WASM port of Needle for browser and embedded deployment
- Dominant language
- Python
- Stars
- 11k
- Forks
- 710
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 25
Description
Hi Cactus team, sharing this in case it's useful for the community.
I built `needle-rs`, an independent Rust runtime for Needle. The architecture
is implemented from your published model card; no upstream code is copied.
The goal is to deploy Needle in contexts where the JAX implementation can't
reach: browsers, Cloudflare Workers, Node.js, no_std embedded, and C-FFI from
any language.
- Repo: https://github.com/Geekgineer/needle-rs
- Live demo (model runs in-browser): https://needle-rs.pages.dev
- HF weights (format-converted, base_model set to your repo):
https://huggingface.co/Abdalrahman/needle-rs-safetensors
Some details that might be useful for anyone evaluating it:
- 258 KB WASM after wasm-opt -Oz; 533 KB native CLI; 557 KB C cdylib
- INT4 group-wise quantization bit-identical to your quantize.py
- 560 inference examples produce token-exact identical output to the
JAX reference, enforced in CI
- Runs on x86_64 (runtime AVX2 dispatch via CPUID), aarch64 (NEON
unconditional), and wasm32 (scalar fallback)
This is explicitly positioned as complementary to your work, not competitive
with it. Your engine targets mobile and NPUs with hand-tuned ARM SIMD; this
targets everywhere else. The two stacks address different deployment
contexts.
I noticed "Graph/Engine→Rust" on your May 2026 roadmap — happy to coordinate
or hand off any of this if it's useful, or to keep it as an independent
community port. Either works.
Thanks for releasing Needle under MIT. The decision to open the weights,
training code, and dataset generation pipeline is what made building this
runtime possible. Full credit and citation throughout the repo.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.