Owner

cactus-compute

3 indexed repositories · View on GitHub

needle
14MB foundation model for tiny devices; phones, wearables, smart home, and robots.

Python · 11006 stars
cactus
Quantization, kernels, runtime and inference engine for mobiles, wearables, smart home and robots.

C++ · 6013 stars
cactus-hybrid
On-device models that know when they're wrong: every answer carries a confidence score for cloud handoff.

Shell · 265 stars

Open All

Three-way routing between E2B/E4B or Qwen3.5 4B/9B, 28B-A4B or Qwen3.5B-35B-A3B, and Cloud Open

cactus-compute/cactus-hybrid#3 · 1 comment · 1 reaction · 0 assignees · Indexed Sep 14, 2026
CFG Implementation Open

hard

cactus-compute/cactus#376 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Add custom vocabulary / hotword biasing for transcription Open

For Cactus Jacks hard

cactus-compute/cactus#396 · 2 comments · 2 reactions · 1 assignee · Indexed Sep 14, 2026
Cactus VAD is 15% slower than ONNX runtime Open

cactus-compute/cactus#399 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Hexagon HVX Open

For Cactus Jacks

cactus-compute/cactus#414 · 2 comments · 0 reactions · 2 assignees · Indexed Sep 14, 2026
FunctionGemma FP16 issues Open

cactus-compute/cactus#503 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 14, 2026
[Bug] Gemma fails on multi tool call because of wrong stop token and bug in logit biasing Open

cactus-compute/cactus#509 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Publish Python bindings to PyPi Open

cactus-compute/cactus#514 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
llguidance support Open

cactus-compute/cactus#553 · 5 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Illegal instruction on Raspberry Pi 4 (ARMv8) Open

cactus-compute/cactus#561 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Gemma 4 and Qwen TTS support Open

cactus-compute/cactus#568 · 3 comments · 1 reaction · 0 assignees · Indexed Sep 14, 2026
Split native LLM ownership into Model and Context Open

cactus-compute/cactus#572 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
f8afc46 regresses Gemma tokenizer loading: tab tokens in vocab.txt cause std::stof exception during init Open

cactus-compute/cactus#577 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
cactus_complete with Gemma 4 E4B + audio input throws unordered_map::at: key not found Open

cactus-compute/cactus#584 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Gemma 4 vision_encoder.mlpackage not working as intended Open

cactus-compute/cactus#592 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Expected single token encoding for <|image_start|> for LFM2-VL-450M Open

cactus-compute/cactus#597 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 14, 2026
cactus_complete returns -1 on second invocation (iOS xcframework, Gemma 4 E2B int4) Open

cactus-compute/cactus#599 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 14, 2026
'Failed to build cactus library' on WSL2 (x86_64) Open

cactus-compute/cactus#655 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Feature: Add SenseVoice/Paraformer ASR models for on-device speech recognition Open

cactus-compute/cactus#678 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Gemma byte-fallback tokens are not reassembled into UTF-8 Open

cactus-compute/cactus#684 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Gemma4 tool-calling emits declaration-format echoes instead of <|tool_call> (function_calls always empty) Open

bug

cactus-compute/cactus#693 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Go bindings? Open

cactus-compute/cactus#720 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 14, 2026
How to reproduce the Needle benchmark numbers on Cactus? Open

cactus-compute/cactus#749 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Does past conversations can't delete？ Open

cactus-compute/cactus#751 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
كيف تحافظ على كفاءة جهاز تنقية الهواء لسنوات؟ Open

cactus-compute/cactus#753 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 14, 2026
`parse_messages_json` drops a message's `content` when the JSON key order is `content` before `role` Open

cactus-compute/cactus#757 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
fix: resolve Python FFI buffer limits and single-quote escaping in tool calling Open

cactus-compute/cactus#760 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Finetuned Needle checkpoint converts to a CQ4 bundle that emits token garbage (stock converts fine) Open

cactus-compute/cactus#767 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 14, 2026
iOS/macOS xcframework: `import cactus` fails as shipped — bundled cactus_engine.h includes cactus_graph.h, which is not in the framework Open

cactus-compute/cactus#771 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 14, 2026
parse_messages_json is key-order-sensitive: `{"content":…,"role":…}` silently generates from an empty user turn Open

cactus-compute/cactus#772 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Long-context KeyDiff KV-cache compaction crashes with EXC_BAD_ACCESS in cactus::kvcompress::rerope_recent_int8 (multi-turn session, ~900 decode tokens) Open

cactus-compute/cactus#777 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 14, 2026
transcribe() returns degenerate repeated tokens instead of speech — whisper-small, Android arm64 Open

cactus-compute/cactus#780 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 14, 2026
RN Android indexAdd skips the input validation iOS performs — ragged embeddings cause an out-of-bounds read Open

cactus-compute/cactus#781 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Repository clone is over 1 GB due to generated binaries in history Open

cactus-compute/cactus#786 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Proposal: add an x86_64 portability baseline Open

cactus-compute/cactus#789 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Shared weights are mmap'd once per component graph Open

cactus-compute/cactus#793 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
cactus-android AAR (1.4.3-beta) cannot load currently published HF model weights (CACT weight format mismatch) Open

cactus-compute/cactus#795 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
16KB page size Open

cactus-compute/cactus#798 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
main fails to compile: duplicate nested 'namespace GraphFile' in cactus-graph/src/io.cpp (introduced by #794) Open

cactus-compute/cactus#801 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
v2.1.0: cactus_complete with an image never returns — Metal backend deadlocks in waitUntilCompleted during chunked media prefill (macOS + iOS) Open

cactus-compute/cactus#802 · 0 comments · 1 reaction · 0 assignees · Indexed Sep 14, 2026
Non-ASCII input corrupted: Python binding escapes to \uXXXX (ensure_ascii) and the engine never decodes it Open

cactus-compute/cactus#803 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
BPE byte-level mapping: byte 0xAD mapped to identity instead of U+0143 (GPT-2 bytes_to_unicode) — CJK chars containing 0xAD tokenize to <unk> Open

cactus-compute/cactus#806 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Needle available on Android possibility Open

cactus-compute/needle#17 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
from jax import linear_util as lu ImportError: cannot import name 'linear_util' Open

cactus-compute/needle#24 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 14, 2026
the model out of box seems not very good Open

cactus-compute/needle#25 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Why re-implement constrained decoding? Open

cactus-compute/needle#28 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Can't use in Lm Studio or Unsloth Open

cactus-compute/needle#30 · 1 comment · 1 reaction · 0 assignees · Indexed Sep 14, 2026
setup script false positive TPU detection Open

cactus-compute/needle#31 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 14, 2026
[Discussion] Pure-Rust + WASM port of Needle for browser and embedded deployment Open

cactus-compute/needle#32 · 11 comments · 16 reactions · 0 assignees · Indexed Sep 14, 2026
How to reproduce the Needle benchmark numbers on Cactus? Open

cactus-compute/needle#33 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026

cactus-compute

Get new issues in your inbox