apple / apple/coreai-models

ANECCompileOffline() failed: OSStatus=0, aneCompileStatus=1 with an empty ErrorList — coreai-build exits 0 and writes a GPU-only .aimodelc for a linear-INT4 static LLM (follow-up to #55)

Open
#205 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Swift
Stars
2.1k
Forks
202
Avg merge
2d 8h
Merged PRs (30d)
55

Description

## Summary

Follow-up to #55, filed separately as invited there ([comment](https://github.com/apple/coreai-models/issues/55#issuecomment-5443603467)). Thanks again for the quick retest cycle on that thread — the SIGSEGV fix holds in every run below.

Compiling the same linear blockwise-INT4 static LLM program from #55 with `--preferred-compute neural-engine` now **exits 0 while ANE compilation fails internally**, producing a **GPU-only `.aimodelc` (0 `*ANE_region*` entries)**. The tool prints an `Error:` block to stderr, but the exit code stays 0, the block's `ErrorList` is empty, and the `.aimodelc` is written without further notice.

The ask: could the ANE compile failure be reflected in the exit code, or in a populated `ErrorList` naming the failing region? Either would make the fallback detectable. Whether linear INT4 can be legalized for the ANE at all is a separate question I'm not raising here — this issue is only about the missing failure signal.

Excerpt (timestamp line and temp path elided):

```
Error:

ANECCompileOffline() failed: OSStatus=0, aneCompileStatus=1, statusdict={
CompiledInputSourceFileName = ".../extend_1024_16_..._ANE_region_1_0.bc.mlir";
ErrorList = (
);
NetworkStatusList = (
);
}
```

## Why it matters

#55's crash was loud; this is the same configuration failing silently. A user who exports linear-INT4 weights gets a clean exit and a loadable model, but everything runs on the GPU. No machine-checkable signal — exit code or error entry — says ANE compilation failed. In my runs the `Error:` block sat inside ~160–525 KB of MLIR warning output; the only other failure-specific trace is six `failed: ANE regionCall op not found` warnings in the same stream (the passing control prints no warnings at all). A user who finds either still faces an empty `ErrorList`.

## Reproduction

Identical to #55 — the export script and compile command are in that issue's body. One adaptation for current versions: `coreai-core 1.0.0b2`'s `save_asset` needs a `pathlib.Path` instead of a `str`.

Reproduced 4/4:

- 2026-08-27, three compiles: two of the artifact from my original checkout (one a solo rerun), one of an artifact regenerated with a pristine clone of `main` @ f43b6da.
- 2026-08-28, one more compile of the original artifact while writing this issue.

All four print the same `ANECCompileOffline()` failure at the same site (`extend_1024_16_*_ANE_region_1_0.bc.mlir`) and exit 0. The produced bundle is GPU-only: `find qwen3_0_6b_ios_int4linear.h18p.aimodelc -name '*ANE_region*' | wc -l` → 0, and its `main-h18p-delegates/` holds only `MPSGraph`.

## Control

The palettized control from #55 still compiles clean on the same toolchain: exit 0, no ANEC failures, and the same `find` count is **31** — recompiled 2026-08-28 back-to-back with the failing run, matching #55's June control. The toolchain still emits ANE fine; the failure is specific to the linear-INT4 program. Disk space is ruled out (456 GiB free).

## Environment

- macOS 27.0 (26A5416b), Apple M4 Max (Mac16,9)
- coreai-build 3600.82.1 (MetalToolchain v27.1.5237.12)
- coreai-torch 0.4.2, coreai-core 1.0.0b2, coreai-opt 0.2.1
- Target arch h18p

Full stderr logs (2026-08-28 failing run + control, plus the pristine-`main` run; home directory redacted in the failing-run log): https://gist.github.com/john-rocky/99082c8cdb0221a594742df5cb2f6756

Thanks again — happy to rerun on any future seed, and can share the failing `.aimodel` directly if useful.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the export script and compile command referenced in issue #55, then compare the failing and palettized control runs using the linked stderr logs. Verify the empty ErrorList, exit code 0, and absence of ANE_region entries; done means the ANE failure is machine-detectable through a nonzero exit or a populated error entry.

Written by the indexing model from the issue text.

Assessment

Tech stack
macos, python, swift
Domain
build-system, devtools, machine-learning
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.