Failed to acquire the source buffer for the ViewOp — MPSGraph scratch heap overflows on any S>1 prefill of Gemma 4 E2B on iOS 27 (iOS face of #27, still on 24A5418b)
- Dominant language
- Swift
- Stars
- 2.1k
- Forks
- 202
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 55
Description
## Summary
The iOS runtime aborts on the first multi-token prefill chunk of a Gemma 4 E2B decode bundle: the per-encode intermediates overflow the graph's MPSGraph scratch heap. This is the iOS manifestation reported in my 2026-07-16 comment on #27; that issue was closed after the macOS overflow could no longer be replicated, so — as invited there — filing the iOS face separately. It still reproduces byte-identically on the current beta with a freshly re-exported and re-AOT'd bundle.
## Environment
- iPhone 17 Pro (h18p), iOS 27.0 (**24A5418b**)
- Host: macOS 27.0 (26A5416b), `coreai-build` **3600.82.1** (MetalToolchain v27.1.5237.12)
- `coreai-torch` 0.4.2, `coreai-core` 1.0.0b2, `coreai-opt` 0.2.1
- Bundle: Gemma 4 E2B QAT int4-linear `tbl` decode (in-graph PLE gather + head), multifunction export — `main` = S=1 decode + `prefill` = static S=32 chunk (shared weights), AOT-compiled for h18p today (not a stale artifact)
## Result (2/2 runs)
```
allocateMTLBufferFromMTLHeap: offset 98816 + size 98304 exceeds heap total 145920
.../MPSRuntime/Operations/GPUMemrefOps.mm:707: failed assertion `Failed to acquire the source buffer for the ViewOp'
App terminated due to signal 6.
```
98304 = 32·1536·2, the S=32 fp16 hidden-state intermediate. Offsets, sizes, and heap total are identical to the July run on 24A5380h; the assertion has moved from line 700 to 707 since.
## Why chunk size cannot work around it (July sweep, same bundle family)
| chunk `S` | abort |
|---|---|
| 64 | `offset 512 + size 196608 exceeds heap total 145920` — the 64·1536·2 hidden buffer |
| 32 | `offset 98816 + size 98304` — two concurrent 32·1536·2 buffers |
| 16 | clears the hidden buffers, then a ~560 KB attention intermediate overflows the same heap |
The overflowing allocation scales with query width `S`, so only S=1 stays under the heap — batched prefill is unavailable, and a 1024-token prompt degrades to per-token processing (prefill throughput ≈ decode throughput).
## Scope note
A different multifunction bundle with a static S=64 `prefill` function (Qwen3-VL-2B) runs clean on the same phone and build — this looks like per-graph under-sizing of the scratch heap for this graph family, not a blanket S>1 limitation.
Happy to attach the device `.ips` crash reports from today's aborts.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the Gemma 4 E2B QAT int4-linear multifunction bundle's `prefill` entry point on an iPhone 17 Pro running iOS 27. Reproduce the S=32, S=16, and S=64 failures and compare them with the clean Qwen3-VL-2B S=64 `prefill` bundle. Done means Gemma prefill no longer aborts from the MPSGraph scratch heap under the reported configuration.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- ios, swift
- Domain
- ai, mobile-dev
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100