cactus-compute / cactus-compute/cactus

v2.1.0: cactus_complete with an image never returns — Metal backend deadlocks in waitUntilCompleted during chunked media prefill (macOS + iOS)

Open
#802 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
C++
Stars
6k
Forks
501
Avg merge
1d 18h
Merged PRs (30d)
4

Description

# v2.1.0: `cactus_complete` with an image never returns — Metal backend deadlocks in `waitUntilCompleted` during chunked media prefill (macOS + iOS)

## Summary

On v2.1.0, any completion with an image attached hangs forever inside `cactus_complete`. The engine loads the model, encodes the image successfully, tokenizes the prompt, submits the media chunked-prefill work to Metal, and then blocks permanently in `[MTLCommandBuffer waitUntilCompleted]` at 0% CPU. Reproduced identically on **macOS (Apple M5 Pro, macOS 26.4.1)** and **iOS (iPhone 15 Pro)**, so it is not device-specific. The **CPU backend does not block** (it computes actively on all cores through the same prefill), which isolates the fault to the Metal path.

Text-only completion on the same model/engine works normally (≈104 tok/s decode on the iPhone), and image *encoding* alone works (≈250 ms, dim 768). Only image→text generation is affected — which makes every VLM chat use case unusable on v2.1.0's Metal backend.

## Environment

- cactus 2.1.0 (Homebrew CLI; engine also rebuilt from the `v2.1.0` git tag — same behavior)
- Model: prebuilt `Cactus-Compute/LFM2-VL-450M` cq4 bundle (weights tag `v2.0`)
- macOS 26.4.1 (Apple M5 Pro), Xcode 26.6; iPhone 15 Pro (iPhone16,1)
- Cloud handoff and telemetry disabled (`--no-cloud-handoff --no-cloud-tele`)

## Reproduction (macOS)

```bash
cactus run Cactus-Compute/LFM2-VL-450M \
--image any.jpg --prompt "Describe this image in one sentence." \
--max-new-tokens 30 --no-cloud-handoff --no-cloud-tele
```

Model loads, the prompt and image line print, then no output. The engine process (`cactus/bin/run`) consumes ~2 s of CPU (load + image encode) and then sits at 0% CPU indefinitely (observed >10 min on iOS, >4 min on macOS; expected ~4 s). With engine DEBUG logging enabled, the last lines emitted are:

```
├─ Warmup: image_embed=256.13ms (dim 768)
[log 0] complete: Prompt tokens: 1043, max_tokens: 100
```

Nothing follows — no first-token/warmup line, no error.

## Stack of the hung engine (macOS, `sample` of `cactus/bin/run`)

Main thread, 100% of samples:

```
main
cactus_complete
cactus::engine::Model::prefill_with_media(...)
cactus::engine::Model::run_chunk_prefill_path(...)
CactusGraph::execute(...)
cactus_metal_session_sync()
-[_MTLCommandBuffer waitUntilCompleted]
_pthread_cond_wait ← blocked forever
```

All other threads are idle pool/telemetry waiters. The submitted command buffer never completes and no error surfaces.

## Control: CPU backend does not block

The identical command with `--backend cpu` behaves completely differently: the engine computes at full multi-core utilization through the prefill instead of parking at 0% CPU. The deadlock is specific to the Metal path (`cactus_metal_session_sync`).

## Related harness gap (why `cactus benchmark` never catches this)

`vlm_benchmark` hard-fails on vision-only models before reaching image→text decode: `[audio_embed] Exception: Model has no audio_encoder component` aborts the suite right after the image-embed warmup. So the exact call that deadlocks is never exercised by the shipped benchmark. (Separately, `transcribe_benchmark` reports `decode_tps` around 10,712,962 tok/s — a timing divide-by-zero — but that's cosmetic by comparison.)

Contributor guide

Open the contributing guide

Research direction

Reproduce the hang with the provided image command, then compare it with the CPU backend. Trace cactus_complete through prefill_with_media, run_chunk_prefill_path, CactusGraph::execute, and cactus_metal_session_sync; use the reported waitUntilCompleted stack as the starting point. Done means image-to-text completion returns on Metal without blocking, with the existing text-only and CPU controls still working.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
ai, mobile-dev
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.