huggingface / huggingface/swift-chat

Failed to build the model after switching to .cpuAndNeuralEngine

Open
#31 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Swift
Stars
597
Forks
49
PR merge metrics
No merged PRs in 30d

Description

**Model Used :** StatefulMistral7BInstructInt4.mlpackage from apple/mistral-coreml
**Hardware :** M3 MacBook Air
**macOS Version:** 15.1.1

The problem occured after changing
```swift
return try LanguageModel.loadCompiled(url: lastURL, computeUnits: .cpuAndGPU)
```
to
```swift
return try LanguageModel.loadCompiled(url: lastURL, computeUnits: .cpuAndNeuralEngine)
```
in ModelLoader.swift line 45

The app builds and runs normally, but doesn't load the model.

Before that single change, the app was working without problems, and when reverting the change it worked again.
I tried to remove the files at `/Users/myuser/Library/Application Support/hf-compiled-transformers/StatefulMistral7BInstructInt4.mlmodelc/model.mil` but it did nothing.

Here is the error message that appeared in the terminal :
`No model could be loaded: Error Domain=com.apple.CoreML Code=0 "Failed to build the model execution plan using a model architecture file '/Users/myuser/Library/Application Support/hf-compiled-transformers/StatefulMistral7BInstructInt4.mlmodelc/model.mil' with error code: 14." UserInfo={NSLocalizedDescription=Failed to build the model execution plan using a model architecture file '/Users/myuser/Library/Application Support/hf-compiled-transformers/StatefulMistral7BInstructInt4.mlmodelc/model.mil' with error code: 14.}`

I can provide more info if needed

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with ModelLoader.swift line 45 and compare the cpuAndGPU and cpuAndNeuralEngine values passed to LanguageModel.loadCompiled. Reproduce the failure with StatefulMistral7BInstructInt4.mlpackage on the reported macOS and hardware, then verify that the model loads successfully with cpuAndNeuralEngine without the CoreML execution-plan error.

Written by the indexing model from the issue text.

Assessment

Tech stack
macos, swift
Domain
desktop, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.