apple / apple/coreai-models

Failed to acquire the source buffer for the ViewOp — MPSGraph scratch heap overflows on any S>1 prefill of Gemma 4 E2B on iOS 27 (iOS face of #27, still on 24A5418b)

Open
#201 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Swift
Stars
2.1k
Forks
202
Avg merge
2d 8h
Merged PRs (30d)
55

Description

## Summary

The iOS runtime aborts on the first multi-token prefill chunk of a Gemma 4 E2B decode bundle: the per-encode intermediates overflow the graph's MPSGraph scratch heap. This is the iOS manifestation reported in my 2026-07-16 comment on #27; that issue was closed after the macOS overflow could no longer be replicated, so — as invited there — filing the iOS face separately. It still reproduces byte-identically on the current beta with a freshly re-exported and re-AOT'd bundle.

## Environment

- iPhone 17 Pro (h18p), iOS 27.0 (**24A5418b**)
- Host: macOS 27.0 (26A5416b), `coreai-build` **3600.82.1** (MetalToolchain v27.1.5237.12)
- `coreai-torch` 0.4.2, `coreai-core` 1.0.0b2, `coreai-opt` 0.2.1
- Bundle: Gemma 4 E2B QAT int4-linear `tbl` decode (in-graph PLE gather + head), multifunction export — `main` = S=1 decode + `prefill` = static S=32 chunk (shared weights), AOT-compiled for h18p today (not a stale artifact)

## Result (2/2 runs)

```
allocateMTLBufferFromMTLHeap: offset 98816 + size 98304 exceeds heap total 145920
.../MPSRuntime/Operations/GPUMemrefOps.mm:707: failed assertion `Failed to acquire the source buffer for the ViewOp'
App terminated due to signal 6.
```

98304 = 32·1536·2, the S=32 fp16 hidden-state intermediate. Offsets, sizes, and heap total are identical to the July run on 24A5380h; the assertion has moved from line 700 to 707 since.

## Why chunk size cannot work around it (July sweep, same bundle family)

| chunk `S` | abort |
|---|---|
| 64 | `offset 512 + size 196608 exceeds heap total 145920` — the 64·1536·2 hidden buffer |
| 32 | `offset 98816 + size 98304` — two concurrent 32·1536·2 buffers |
| 16 | clears the hidden buffers, then a ~560 KB attention intermediate overflows the same heap |

The overflowing allocation scales with query width `S`, so only S=1 stays under the heap — batched prefill is unavailable, and a 1024-token prompt degrades to per-token processing (prefill throughput ≈ decode throughput).

## Scope note

A different multifunction bundle with a static S=64 `prefill` function (Qwen3-VL-2B) runs clean on the same phone and build — this looks like per-graph under-sizing of the scratch heap for this graph family, not a blanket S>1 limitation.

Happy to attach the device `.ips` crash reports from today's aborts.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the Gemma 4 E2B QAT int4-linear multifunction bundle's `prefill` entry point on an iPhone 17 Pro running iOS 27. Reproduce the S=32, S=16, and S=64 failures and compare them with the clean Qwen3-VL-2B S=64 `prefill` bundle. Done means Gemma prefill no longer aborts from the MPSGraph scratch heap under the reported configuration.

Written by the indexing model from the issue text.

Assessment

Tech stack
ios, swift
Domain
ai, mobile-dev
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.