microsoft / microsoft/onnxruntime-genai
How to build for iOS
- Dominant language
- C++
- Stars
- 1.1k
- Forks
- 354
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 85
Description
Hello,
I am trying to build onnxruntime-genai for iOS using this tutorial https://github.com/Azure-Samples/Phi-3MiniSamples/blob/main/ios/README.md. But i think it is now out of date as the is no branch to checkout
```
git checkout yguo/ios-build-genai
python3 build.py --parallel --build_dir ./build_ios_simulator --ios --ios_sysroot iphoneos --osx_arch arm64 --apple_deployment_target 17.4 --cmake_generator Xcode
```
as such it fails to build the genAI library for iOS. The above tutorial does have the .dylib in the example project (but i dont think we can release apps to the app store which have dynamic libraries) -> i have been trying to get it to work as a framework but it is very buggy and slow and wont work with smaller models like Qwen0.7B - i get the error "model not supported". I feel this is because the dynamic library i am using for Gen-AI is from July 2024? I did try to build the latest library using this code
https://gist.github.com/rolfbjarne/ded0063575bcb5c0cf14ffdc9aa197b7
but it doesnt work at all.
Any help is appreciated for getting the latest library working in iOS, GenAI is so cool and will make deploying LLM on iPhones super easy for novice coders like me.
Thank you in advance.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the iOS README tutorial, the referenced build.py command, and the linked build gist; verify which branch and library format are currently supported. Reproduce the iOS build failure and the “model not supported” error with the mentioned example, then document a current working path and its model and App Store constraints.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cmake, ios, python
- Domain
- build-system, mobile-dev
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100