cactus-compute / cactus-compute/cactus

cactus-android AAR (1.4.3-beta) cannot load currently published HF model weights (CACT weight format mismatch)

Open
#795 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
6k
Forks
501
Avg merge
1d 18h
Merged PRs (30d)
4

Description

## Summary
The published Android AAR `com.cactuscompute:cactus-android:1.4.3-beta` (latest on Maven Central) fails to initialize ANY model downloaded from the Cactus-Compute HuggingFace org. `cactus_init` returns 0 / null for `qwen3-0.6b-int4`, `lfm2.5-350m-int4`, `gemma-3-270m-it-int4/int8`, etc.

## Root cause (verified by inspecting the weight files and engine sources)
All currently published quantized weight files (`weights/*-int4.zip`, `weights/*-int8.zip`, `L*.zip`) use a CACT tensor header with `flags=9` = `FLAG_HAS_SCALES (1<<0)` | `FLAG_INTERLEAVED (1<<3)`, written by `python/cactus/convert/cactus_adapters/tensor_io.py` (INT4: precision=3, INT8: precision=0).

However the engine shipped inside the 1.4.3-beta AAR only understands the older flags (`FLAG_ORTHOGONAL_ROTATION 1<<1`, `FLAG_INTERLEAVED_4ROW 1<<2`, `FLAG_EXTENDED_SHAPE 1<<4`) and has no FLAG_HAS_SCALES / FLAG_INTERLEAVED handling, so weight parsing fails at model load time.

The current engine source (main, v2.1.0) still only checks FLAG_INTERLEAVED_4ROW in cactus-graph/src/io.cpp — the FLAG_INTERLEAVED (1<<3) format written by the current tensor_io.py is not read anywhere in the C++ code I could find.

## Impact
Third-party Android apps integrating the official AAR cannot run any on-device model. The SDK's own Supabase model catalog cannot be used either (anon key is shipped redacted in the AAR).

## Request
- Publish an updated `cactus-android` AAR built from the current engine (v2.x) that reads the current weight format, OR
- Publish the model weights in the format the 1.4.3-beta AAR can read, OR
- Confirm the correct pairing of AAR version <-> published weights, and document it.

Happy to help test a new AAR.

Contributor guide

Open the contributing guide

Research direction

Start by comparing the CACT flags in python/cactus/convert/cactus_adapters/tensor_io.py with the parsing in cactus-graph/src/io.cpp and the 1.4.3-beta Android AAR. Verify which engine and weight versions are compatible, then validate model initialization with the cited Hugging Face weights. Done means compatible published artifacts or a documented AAR-to-weights pairing that loads the affected models.

Written by the indexing model from the issue text.

Assessment

Tech stack
android, cpp
Domain
ai, mobile
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.