huggingface / huggingface/swift-chat
Failed to build the model after switching to .cpuAndNeuralEngine
- Dominant language
- Swift
- Stars
- 597
- Forks
- 49
- PR merge metrics
- No merged PRs in 30d
Description
**Model Used :** StatefulMistral7BInstructInt4.mlpackage from apple/mistral-coreml
**Hardware :** M3 MacBook Air
**macOS Version:** 15.1.1
The problem occured after changing
```swift
return try LanguageModel.loadCompiled(url: lastURL, computeUnits: .cpuAndGPU)
```
to
```swift
return try LanguageModel.loadCompiled(url: lastURL, computeUnits: .cpuAndNeuralEngine)
```
in ModelLoader.swift line 45
The app builds and runs normally, but doesn't load the model.
Before that single change, the app was working without problems, and when reverting the change it worked again.
I tried to remove the files at `/Users/myuser/Library/Application Support/hf-compiled-transformers/StatefulMistral7BInstructInt4.mlmodelc/model.mil` but it did nothing.
Here is the error message that appeared in the terminal :
`No model could be loaded: Error Domain=com.apple.CoreML Code=0 "Failed to build the model execution plan using a model architecture file '/Users/myuser/Library/Application Support/hf-compiled-transformers/StatefulMistral7BInstructInt4.mlmodelc/model.mil' with error code: 14." UserInfo={NSLocalizedDescription=Failed to build the model execution plan using a model architecture file '/Users/myuser/Library/Application Support/hf-compiled-transformers/StatefulMistral7BInstructInt4.mlmodelc/model.mil' with error code: 14.}`
I can provide more info if needed
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with ModelLoader.swift line 45 and compare the cpuAndGPU and cpuAndNeuralEngine values passed to LanguageModel.loadCompiled. Reproduce the failure with StatefulMistral7BInstructInt4.mlpackage on the reported macOS and hardware, then verify that the model loads successfully with cpuAndNeuralEngine without the CoreML execution-plan error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- macos, swift
- Domain
- desktop, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100