cactus-compute / cactus-compute/cactus
v2.1.0: cactus_complete with an image never returns — Metal backend deadlocks in waitUntilCompleted during chunked media prefill (macOS + iOS)
- Dominant language
- C++
- Stars
- 6k
- Forks
- 501
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 4
Description
# v2.1.0: `cactus_complete` with an image never returns — Metal backend deadlocks in `waitUntilCompleted` during chunked media prefill (macOS + iOS)
## Summary
On v2.1.0, any completion with an image attached hangs forever inside `cactus_complete`. The engine loads the model, encodes the image successfully, tokenizes the prompt, submits the media chunked-prefill work to Metal, and then blocks permanently in `[MTLCommandBuffer waitUntilCompleted]` at 0% CPU. Reproduced identically on **macOS (Apple M5 Pro, macOS 26.4.1)** and **iOS (iPhone 15 Pro)**, so it is not device-specific. The **CPU backend does not block** (it computes actively on all cores through the same prefill), which isolates the fault to the Metal path.
Text-only completion on the same model/engine works normally (≈104 tok/s decode on the iPhone), and image *encoding* alone works (≈250 ms, dim 768). Only image→text generation is affected — which makes every VLM chat use case unusable on v2.1.0's Metal backend.
## Environment
- cactus 2.1.0 (Homebrew CLI; engine also rebuilt from the `v2.1.0` git tag — same behavior)
- Model: prebuilt `Cactus-Compute/LFM2-VL-450M` cq4 bundle (weights tag `v2.0`)
- macOS 26.4.1 (Apple M5 Pro), Xcode 26.6; iPhone 15 Pro (iPhone16,1)
- Cloud handoff and telemetry disabled (`--no-cloud-handoff --no-cloud-tele`)
## Reproduction (macOS)
```bash
cactus run Cactus-Compute/LFM2-VL-450M \
--image any.jpg --prompt "Describe this image in one sentence." \
--max-new-tokens 30 --no-cloud-handoff --no-cloud-tele
```
Model loads, the prompt and image line print, then no output. The engine process (`cactus/bin/run`) consumes ~2 s of CPU (load + image encode) and then sits at 0% CPU indefinitely (observed >10 min on iOS, >4 min on macOS; expected ~4 s). With engine DEBUG logging enabled, the last lines emitted are:
```
├─ Warmup: image_embed=256.13ms (dim 768)
[log 0] complete: Prompt tokens: 1043, max_tokens: 100
```
Nothing follows — no first-token/warmup line, no error.
## Stack of the hung engine (macOS, `sample` of `cactus/bin/run`)
Main thread, 100% of samples:
```
main
cactus_complete
cactus::engine::Model::prefill_with_media(...)
cactus::engine::Model::run_chunk_prefill_path(...)
CactusGraph::execute(...)
cactus_metal_session_sync()
-[_MTLCommandBuffer waitUntilCompleted]
_pthread_cond_wait ← blocked forever
```
All other threads are idle pool/telemetry waiters. The submitted command buffer never completes and no error surfaces.
## Control: CPU backend does not block
The identical command with `--backend cpu` behaves completely differently: the engine computes at full multi-core utilization through the prefill instead of parking at 0% CPU. The deadlock is specific to the Metal path (`cactus_metal_session_sync`).
## Related harness gap (why `cactus benchmark` never catches this)
`vlm_benchmark` hard-fails on vision-only models before reaching image→text decode: `[audio_embed] Exception: Model has no audio_encoder component` aborts the suite right after the image-embed warmup. So the exact call that deadlocks is never exercised by the shipped benchmark. (Separately, `transcribe_benchmark` reports `decode_tps` around 10,712,962 tok/s — a timing divide-by-zero — but that's cosmetic by comparison.)
Contributor guide
Research direction
Reproduce the hang with the provided image command, then compare it with the CPU backend. Trace cactus_complete through prefill_with_media, run_chunk_prefill_path, CactusGraph::execute, and cactus_metal_session_sync; use the reported waitUntilCompleted stack as the starting point. Done means image-to-text completion returns on Metal without blocking, with the existing text-only and CPU controls still working.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- ai, mobile-dev
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100