cactus-compute / cactus-compute/needle

[Discussion] Pure-Rust + WASM port of Needle for browser and embedded deployment

Open
#32 11 comments 16 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
11k
Forks
710
Avg merge
1d 21h
Merged PRs (30d)
25

Description

Hi Cactus team, sharing this in case it's useful for the community.

I built `needle-rs`, an independent Rust runtime for Needle. The architecture
is implemented from your published model card; no upstream code is copied.
The goal is to deploy Needle in contexts where the JAX implementation can't
reach: browsers, Cloudflare Workers, Node.js, no_std embedded, and C-FFI from
any language.

- Repo: https://github.com/Geekgineer/needle-rs
- Live demo (model runs in-browser): https://needle-rs.pages.dev
- HF weights (format-converted, base_model set to your repo):
https://huggingface.co/Abdalrahman/needle-rs-safetensors

Some details that might be useful for anyone evaluating it:

- 258 KB WASM after wasm-opt -Oz; 533 KB native CLI; 557 KB C cdylib
- INT4 group-wise quantization bit-identical to your quantize.py
- 560 inference examples produce token-exact identical output to the
JAX reference, enforced in CI
- Runs on x86_64 (runtime AVX2 dispatch via CPUID), aarch64 (NEON
unconditional), and wasm32 (scalar fallback)

This is explicitly positioned as complementary to your work, not competitive
with it. Your engine targets mobile and NPUs with hand-tuned ARM SIMD; this
targets everywhere else. The two stacks address different deployment
contexts.

I noticed "Graph/Engine→Rust" on your May 2026 roadmap — happy to coordinate
or hand off any of this if it's useful, or to keep it as an independent
community port. Either works.

Thanks for releasing Needle under MIT. The decision to open the weights,
training code, and dataset generation pipeline is what made building this
runtime possible. Full credit and citation throughout the repo.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.