IREE on iOS: The Progress #11732

wangkuiyi · 2023-01-06T05:02:56Z

wangkuiyi
Jan 6, 2023
Collaborator

IREE supports Android https://iree-org.github.io/iree/#support-matrix. It is on its way to supporting iOS.

Build for iOS

Our first step is to enable the build of IREE for iOS. IREE has two sets of building rules, CMake and Bazel. CMake is the recommended one. Bazel is mostly for integration with TensorFlow and Google internal build system.

As kindly explained by @ScottTodd, IREE includes two parts. The compilers run on desktops and servers. They can lower torch.fx.Graph traced from PyTorch programs or JAXPRs traced from JAX programs into vmfb, a flatbuffer containing CPU and GPU code. The runtime runs on devices like an Android phone and executes code in the vmfb.

This discussion #11716 records the progress of enabling the build for iOS. After merging several pull requests by @antiagainst , we were able to build the compilers for macOS and then for iOS (device and simulator).

Step 1. Build the Compiler for macOS

cmake -S . -B build-release/ -GNinja \
  -DCMAKE_BUILD_TYPE=RelWithDebInfo \
  -DCMAKE_INSTALL_PREFIX=build-release/install
  
cmake --build build-release/ --target install

Step 2.1. Build the Runtime for the iOS Simulator

cmake -S . -B build-sim -GNinja \
  -DCMAKE_SYSTEM_NAME=iOS \
  -DCMAKE_OSX_SYSROOT=$(xcodebuild -version -sdk iphonesimulator Path) \
  -DCMAKE_OSX_ARCHITECTURES=arm64 \
  -DCMAKE_SYSTEM_PROCESSOR=arm64 \
  -DCMAKE_OSX_DEPLOYMENT_TARGET=11.0 \
  -DCMAKE_IOS_INSTALL_COMBINED=YES \
  -DIREE_HOST_BINARY_ROOT=$PWD/build-release/install \
  -DCMAKE_INSTALL_PREFIX=build-sim/install \
  -DIREE_BUILD_COMPILER=OFF -DIREE_BUILD_TESTS=OFF -DIREE_BUILD_SAMPLES=OFF

cmake --build build-sim --config Release --target install

In the directory ./build-sim/install/bin, we can see some iOS apps.

./build-sim/install/bin
├── flags_demo.app
│   ├── Info.plist
│   └── flags_demo
...

Jan 5, 2023: The content of Info.plist is incomplete. Details follow.
Jan 7, 2023: @antiagainst fixed this problem in #11742

Step 2.2. Build the Runtime for iOS Devices

cmake -S . -B build-ios -GNinja \
  -DCMAKE_SYSTEM_NAME=iOS \
  -DCMAKE_OSX_SYSROOT=$(xcodebuild -version -sdk iphoneos Path) \
  -DCMAKE_OSX_ARCHITECTURES=arm64 \
  -DCMAKE_SYSTEM_PROCESSOR=arm64 \
  -DCMAKE_OSX_DEPLOYMENT_TARGET=11.0 \
  -DCMAKE_IOS_INSTALL_COMBINED=YES \
  -DIREE_HOST_BINARY_ROOT=$PWD/build-release/install \
  -DCMAKE_INSTALL_PREFIX=build-ios/install \
  -DIREE_BUILD_COMPILER=OFF -DIREE_BUILD_TESTS=OFF -DIREE_BUILD_SAMPLES=OFF

cmake --build build-ios --config Release --target install

The only difference between building for the simulator and devices is the value of -DCMAKE_OSX_SYSROOT.

Run `flags_demo.app` on iOS Simulators

To verify the build result, we want to run an app, say, flags_demo.app, on the iOS simulator.

Like the Android ecosystem offers the Android Debugger, adb, Apple offers a command-line tool, simctl, to manage and operate iOS simulators. Following steps in https://wkyi.quip.com/JYFLAuqPQgNT, I started an iOS simulator on my M1 MacBook Pro.

Step 1. Manually complete the Info.plist file

I manually added the following content into ./build-sim/install/bin/flags_demo.app/Info.plist. Otherwise, the following app install step will fail with error messages complaining the lack of fields like CFBundleVersion.

  <key>CFBundleVersion</key>
  <string>1.0</string>
  <key>CFBundleShortVersionString</key>
  <string>1.0</string>
  <key>CFBundleIdentifier</key>
  <string>com.company.app</string>

Step 2. Install the App

The following command prints the boosted simulator ID.

$ xcrun simctl list devices | grep Booted
    iPhone 14 Pro (50831FFA-58C6-4575-B410-5F4FD7A2DFAA) (Booted)

The following command installs the app to the simulator.

$ xcrun simctl install \
  50831FFA-58C6-4575-B410-5F4FD7A2DFAA \
  ./build-sim/install/bin/flags_demo.app

The following command verifies the installation. Please note that com.company.app is what I added manually into the Info.plist file to identify the app.

$ xcrun simctl listapps \
  50831FFA-58C6-4575-B410-5F4FD7A2DFAA \
  | grep com.company.app
    "com.company.app" =     {
        CFBundleIdentifier = "com.company.app";

Step 3. Run the App

The following command launches the installed flags_demo.app. It redirects the stdout and stderr of the app to the terminal.

$ xcrun simctl launch --console 50831FFA-58C6-4575-B410-5F4FD7A2DFAA com.company.app
FLAG[test_bool] = false
FLAG[test_int32] = 123
FLAG[test_int64] = 555
FLAG[test_float] = 1
FLAG[test_string] = some default
FLAG[test_callback] = 0
ARG(0) = /Users/y/Library/Developer/CoreSimulator/Devices/50831FFA-58C6-4575-B410-5F4FD7A2DFAA/data/Containers/Bundle/Application/9B94890A-AF06-4556-B670-40739EE805FE/flags_demo.app/flags_demo
# --help
# --flagfile=[path]
--test_bool=false
--test_int32=123
--test_int64=555
--test_float=1
--test_string="some default"
--test_callback=0
FLAG[test_bool] = false
FLAG[test_int32] = 123
FLAG[test_int64] = 555
FLAG[test_float] = 1
FLAG[test_string] = some default
FLAG[test_callback] = 0
ARG(0) = /Users/y/Library/Developer/CoreSimulator/Devices/50831FFA-58C6-4575-B410-5F4FD7A2DFAA/data/Containers/Bundle/Application/9B94890A-AF06-4556-B670-40739EE805FE/flags_demo.app/flags_demo
# --help
# --flagfile=[path]
--test_bool=false
--test_int32=123
--test_int64=555
--test_float=1
--test_string="some default"
--test_callback=0
com.company.app: 14386

Run on iPhones

[TBD: use xcrun altool to notarize the app and publish on App Store.]

Run `iree-run-module` on iOS Simulators

I verified this simple Android example (https://iree-org.github.io/iree/building-from-source/android/#running-tools-directly) works for iOS Simulator.

Build IREE as described above.

Run the compiler built for macOS to generate the vmfb.

./build-release/install/bin/iree-compile \
  --iree-hal-target-backends=vmvx \
  samples/models/simple_abs.mlir \
  -o /tmp/simple_abs_vmvx.vmfb

Copy the generated vmfb file into the iOS app bundle of iree-run-module.

cp /tmp/simple_abs_vmvx.vmfb \
   ./build-sim/install/bin/iree-run-module.app/

Manually add the following into the ./build-sim/install/bin/iree-run-module.app/Info.plist file.

We will be able to avoid this hacky step once after we fix CMake build for iOS generates incomplete Info.plist files #11733.
```
  <key>CFBundleVersion</key>
  <string>1.0</string>
  <key>CFBundleShortVersionString</key>
  <string>1.0</string>
  <key>CFBundleIdentifier</key>
  <string>org.iree.runmodule</string>
```
You can use an arbitrary bundle ID other than org.iree.runmodule. The following commands refer to the app by this ID.
Install the app to the iOS simulator.
- I am using my simulator's ID in the following command. You might want to figure out yours. Please refer to this document for iOS simulator management.
- You might want to uninstall an old version of the app if you have one.
```
xcrun simctl uninstall 50831FFA-58C6-4575-B410-5F4FD7A2DFAA "org.iree.runmodule"
xcrun simctl install 50831FFA-58C6-4575-B410-5F4FD7A2DFAA ./build-sim/install/bin/iree-run-module.app
```

Check the path to the installed bundle.

ls $(xcrun simctl get_app_container 50831FFA-58C6-4575-B410-5F4FD7A2DFAA org.iree.runmodule)

Launch the iree-run-module app on the simulator.

xcrun simctl launch --console \
  50831FFA-58C6-4575-B410-5F4FD7A2DFAA \
  org.iree.runmodule \
  --device=local-task \
  --entry_function=abs \
  --function_input="f32=-5" \
  --module_file=$(xcrun simctl get_app_container 50831FFA-58C6-4575-B410-5F4FD7A2DFAA org.iree.runmodule)/simple_abs_vmvx.vmfb

It prints the following content.

EXEC @abs
result[0]: hal.buffer_view
f32=5
EXEC @abs
result[0]: hal.buffer_view
f32=5
org.iree.runmodule: 73860

Note: why it seems that the program ran twice and printed the result twice?

An JAX Example

[TBD: Run something like this example https://github.com/iree-org/iree-jax]

antiagainst · 2023-01-06T18:24:02Z

antiagainst
Jan 6, 2023
Collaborator

This is fantastic writeup, thanks all the great work @wangkuiyi! I'll take a look at the Info.plist issue.

0 replies

antiagainst · 2023-01-09T18:00:01Z

antiagainst
Jan 9, 2023
Collaborator

BTW, @wangkuiyi would you mind to turn the steps into a proper doc like what we have here for Android? Just need to create an ios.md file in parallel to android there; build infra would take care of all steps to publish to GitHub pages. This would provide better visibility to the steps for other folks looking at cross compiling towards iOS. Thanks in advance!

5 replies

wangkuiyi Jan 9, 2023
Collaborator Author

Yes, for sure. @antiagainst

I haven't done that yet because apps like iree-run-module.app can run on the iOS simulator but not yet on iPhones. I'm still working on the codesigning stuff, which is the extra thing that needs to be done for the app to run on devices instead of just the simulator.

Should we wait until it can run on devices? Or, I can make the ios.md file now with steps for running on the simulator, and we can add a section about codesigning and running on devices later.

antiagainst Jan 9, 2023
Collaborator

Code signing is mostly for properly releasing final apps, in which we'd assume IREE is called as a library. The existing IREE binaries is developer tools; not meant to be customer facing in mobile devices. So I think it's fine to add docs covering what we have right now (cross compiling + running on simulators). We can always add more stuff later too. :)

wangkuiyi Jan 9, 2023
Collaborator Author

Make sense! I am drafting the PR of ios.md.

antiagainst Jan 9, 2023
Collaborator

Awesome, thanks!

wangkuiyi Jan 9, 2023
Collaborator Author

Here it is #11758 @antiagainst

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

IREE on iOS: The Progress #11732

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 2 comments 5 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

IREE on iOS: The Progress #11732

wangkuiyi Jan 6, 2023 Collaborator

Build for iOS

Step 1. Build the Compiler for macOS

Step 2.1. Build the Runtime for the iOS Simulator

Step 2.2. Build the Runtime for iOS Devices

Run flags_demo.app on iOS Simulators

Step 1. Manually complete the Info.plist file

Step 2. Install the App

Step 3. Run the App

Run on iPhones

Run iree-run-module on iOS Simulators

An JAX Example

Replies: 2 comments · 5 replies

antiagainst Jan 6, 2023 Collaborator

antiagainst Jan 9, 2023 Collaborator

wangkuiyi Jan 9, 2023 Collaborator Author

antiagainst Jan 9, 2023 Collaborator

wangkuiyi Jan 9, 2023 Collaborator Author

antiagainst Jan 9, 2023 Collaborator

wangkuiyi Jan 9, 2023 Collaborator Author

wangkuiyi
Jan 6, 2023
Collaborator

Run `flags_demo.app` on iOS Simulators

Run `iree-run-module` on iOS Simulators

Replies: 2 comments 5 replies

antiagainst
Jan 6, 2023
Collaborator

antiagainst
Jan 9, 2023
Collaborator

wangkuiyi Jan 9, 2023
Collaborator Author

antiagainst Jan 9, 2023
Collaborator

wangkuiyi Jan 9, 2023
Collaborator Author

antiagainst Jan 9, 2023
Collaborator

wangkuiyi Jan 9, 2023
Collaborator Author