cactus-compute / cactus-compute/cactus
cactus-android AAR (1.4.3-beta) cannot load currently published HF model weights (CACT weight format mismatch)
- Dominant language
- C++
- Stars
- 6k
- Forks
- 501
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 4
Description
## Summary
The published Android AAR `com.cactuscompute:cactus-android:1.4.3-beta` (latest on Maven Central) fails to initialize ANY model downloaded from the Cactus-Compute HuggingFace org. `cactus_init` returns 0 / null for `qwen3-0.6b-int4`, `lfm2.5-350m-int4`, `gemma-3-270m-it-int4/int8`, etc.
## Root cause (verified by inspecting the weight files and engine sources)
All currently published quantized weight files (`weights/*-int4.zip`, `weights/*-int8.zip`, `L*.zip`) use a CACT tensor header with `flags=9` = `FLAG_HAS_SCALES (1<<0)` | `FLAG_INTERLEAVED (1<<3)`, written by `python/cactus/convert/cactus_adapters/tensor_io.py` (INT4: precision=3, INT8: precision=0).
However the engine shipped inside the 1.4.3-beta AAR only understands the older flags (`FLAG_ORTHOGONAL_ROTATION 1<<1`, `FLAG_INTERLEAVED_4ROW 1<<2`, `FLAG_EXTENDED_SHAPE 1<<4`) and has no FLAG_HAS_SCALES / FLAG_INTERLEAVED handling, so weight parsing fails at model load time.
The current engine source (main, v2.1.0) still only checks FLAG_INTERLEAVED_4ROW in cactus-graph/src/io.cpp — the FLAG_INTERLEAVED (1<<3) format written by the current tensor_io.py is not read anywhere in the C++ code I could find.
## Impact
Third-party Android apps integrating the official AAR cannot run any on-device model. The SDK's own Supabase model catalog cannot be used either (anon key is shipped redacted in the AAR).
## Request
- Publish an updated `cactus-android` AAR built from the current engine (v2.x) that reads the current weight format, OR
- Publish the model weights in the format the 1.4.3-beta AAR can read, OR
- Confirm the correct pairing of AAR version <-> published weights, and document it.
Happy to help test a new AAR.
Contributor guide
Research direction
Start by comparing the CACT flags in python/cactus/convert/cactus_adapters/tensor_io.py with the parsing in cactus-graph/src/io.cpp and the 1.4.3-beta Android AAR. Verify which engine and weight versions are compatible, then validate model initialization with the cited Hugging Face weights. Done means compatible published artifacts or a documented AAR-to-weights pairing that loads the affected models.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- android, cpp
- Domain
- ai, mobile
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100