cactus-compute / cactus-compute/cactus

cactus_complete returns -1 on second invocation (iOS xcframework, Gemma 4 E2B int4)

Open
#599 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
6k
Forks
503
Avg merge
1d 18h
Merged PRs (30d)
4

Description

Environment

  • iPhone 17 Pro Max (iOS 26)
  • Cactus iOS xcframework (from cactus-react-native v1.14)
  • Model: gemma-4-e2b-it INT4 (Cactus-Compute/gemma-4-E2B-it)
  • Swift, native app (not React Native)

Bug

cactus_complete returns -1 on the second call to the same model handle. The first call after cactus_init + warm-up always succeeds. Every subsequent call fails.

cactus_transcribe with Parakeet works repeatedly on the same model handle — only LLM completions are affected.

Minimal Repro

let model = try cactusInit(path, nil, false)

// Warm-up — works
let _ = try? cactusComplete(model, "[{\"role\":\"user\",\"content\":\"Hi\"}]", "{\"max_tokens\":1}", nil, nil)

// First real call — works
cactusReset(model)
let r1 = try cactusComplete(model, "[{\"role\":\"user\",\"content\":\"Hello\"}]", "{\"max_tokens\":256}", nil, nil)
// r1 = valid JSON response ✓

// Second call — FAILS (result < 0)
cactusReset(model)
let r2 = try cactusComplete(model, "[{\"role\":\"user\",\"content\":\"Who are you?\"}]", "{\"max_tokens\":256}", nil, nil)
// throws "Completion failed"

Workarounds Attempted (all fail on 2nd call)

  • cactusReset(model) before each call
  • cactusStop(model) + cactusReset(model)
  • No reset at all
  • cactusDestroy + cactusInit (full re-init)
  • Running on Task.detached / serial DispatchQueue
  • Running on @MainActor directly
  • Reducing max_tokens to 1
  • Different message content

Context

Building at the YC Gemma 4 Voice Agents Hackathon. This is blocking our demo — we can only get one inference per app launch. Happy to test a fix immediately if you have one.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the failure using the reported cactusInit, cactusComplete, cactusReset, cactusStop, and cactusDestroy entry points with the Gemma 4 E2B INT4 model in the iOS xcframework. Compare the successful first completion with the second call and check why the same model handle returns -1; done means repeated completions succeed after reset or the documented lifecycle is corrected.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, ios, swift
Domain
ai, mobile-dev
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.