microsoft / microsoft/onnxruntime-genai

How to build for iOS

Open
#1,484 0 comments 0 reactions 0 assignees View on GitHub
platform:mobile
Dominant language
C++
Stars
1.1k
Forks
354
Avg merge
2d 16h
Merged PRs (30d)
85

Description

Hello,

I am trying to build onnxruntime-genai for iOS using this tutorial https://github.com/Azure-Samples/Phi-3MiniSamples/blob/main/ios/README.md. But i think it is now out of date as the is no branch to checkout

```
git checkout yguo/ios-build-genai

python3 build.py --parallel --build_dir ./build_ios_simulator --ios --ios_sysroot iphoneos --osx_arch arm64 --apple_deployment_target 17.4 --cmake_generator Xcode
```

as such it fails to build the genAI library for iOS. The above tutorial does have the .dylib in the example project (but i dont think we can release apps to the app store which have dynamic libraries) -> i have been trying to get it to work as a framework but it is very buggy and slow and wont work with smaller models like Qwen0.7B - i get the error "model not supported". I feel this is because the dynamic library i am using for Gen-AI is from July 2024? I did try to build the latest library using this code

https://gist.github.com/rolfbjarne/ded0063575bcb5c0cf14ffdc9aa197b7

but it doesnt work at all.

Any help is appreciated for getting the latest library working in iOS, GenAI is so cool and will make deploying LLM on iPhones super easy for novice coders like me.

Thank you in advance.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the iOS README tutorial, the referenced build.py command, and the linked build gist; verify which branch and library format are currently supported. Reproduce the iOS build failure and the “model not supported” error with the mentioned example, then document a current working path and its model and App Store constraints.

Written by the indexing model from the issue text.

Assessment

Tech stack
cmake, ios, python
Domain
build-system, mobile-dev
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.