1# Building an ExecuTorch Android Demo App 2 3This is forked from [PyTorch Android Demo App](https://github.com/pytorch/android-demo-app). 4 5This guide explains how to setup ExecuTorch for Android using a demo app. The app employs a [DeepLab v3](https://pytorch.org/hub/pytorch_vision_deeplabv3_resnet101/) model for image segmentation tasks. Models are exported to ExecuTorch using [XNNPACK FP32 backend](tutorial-xnnpack-delegate-lowering.md). 6 7::::{grid} 2 8:::{grid-item-card} What you will learn 9:class-card: card-prerequisites 10* How to set up a build target for Android arm64-v8a 11* How to build the required ExecuTorch runtime with JNI wrapper for Android 12* How to build the app with required JNI library and model file 13::: 14 15:::{grid-item-card} Prerequisites 16:class-card: card-prerequisites 17* Refer to [Setting up ExecuTorch](https://pytorch.org/executorch/stable/getting-started-setup) to set up the repo and dev environment. 18* Download and install [Android Studio and SDK](https://developer.android.com/studio). 19* Supported Host OS: CentOS, macOS Ventura (M1/x86_64). See below for Qualcomm HTP specific requirements. 20* *Qualcomm HTP Only[^1]:* To build and run on Qualcomm's AI Engine Direct, please follow [Building and Running ExecuTorch with Qualcomm AI Engine Direct Backend](build-run-qualcomm-ai-engine-direct-backend.md) for hardware and software pre-requisites. The version we use for this tutorial is 2.19. The chip we use for this tutorial is SM8450. 21::: 22:::: 23 24[^1]: This section applies only if Qualcomm HTP Backend is needed in the app. Same applies to sections with title`Qualcomm Hexagon NPU`. 25 26```{note} 27This demo app and tutorial has only been validated with arm64-v8a [ABI](https://developer.android.com/ndk/guides/abis). 28``` 29 30 31## Build 32 33### Ahead-Of-Time 34 35We generate the model file for the ExecuTorch runtime in Android Demo App. 36 37#### XNNPACK Delegation 38 39For delegating DeepLab v3 to XNNPACK backend, please do the following to export the model: 40 41```bash 42python3 -m examples.xnnpack.aot_compiler --model_name="dl3" --delegate 43mkdir -p examples/demo-apps/android/ExecuTorchDemo/app/src/main/assets/ 44cp dl3_xnnpack_fp32.pte examples/demo-apps/android/ExecuTorchDemo/app/src/main/assets/ 45``` 46 47For more detailed tutorial of lowering to XNNPACK, please see [XNNPACK backend](tutorial-xnnpack-delegate-lowering.md). 48 49#### Qualcomm Hexagon NPU 50 51For delegating to Qualcomm Hexagon NPU, please follow the tutorial [here](build-run-qualcomm-ai-engine-direct-backend.md). 52 53After generating the model, copy the model to `assets` directory. 54 55```bash 56python -m examples.qualcomm.scripts.deeplab_v3 -b build-android -m SM8450 -s <adb_connected_device_serial> 57cp deeplab_v3/dlv3_qnn.pte examples/demo-apps/android/ExecuTorchDemo/app/src/main/assets/ 58``` 59 60### Runtime 61 62We build the required ExecuTorch runtime library to run the model. 63 64#### XNNPACK 65 661. Build the CMake target for the library with XNNPACK backend: 67 68```bash 69export ANDROID_NDK=<path-to-android-ndk> 70export ANDROID_ABI=arm64-v8a 71 72rm -rf cmake-android-out && mkdir cmake-android-out 73 74# Build the core executorch library 75cmake . -DCMAKE_INSTALL_PREFIX=cmake-android-out \ 76 -DCMAKE_TOOLCHAIN_FILE="${ANDROID_NDK}/build/cmake/android.toolchain.cmake" \ 77 -DANDROID_ABI="${ANDROID_ABI}" \ 78 -DEXECUTORCH_BUILD_XNNPACK=ON \ 79 -DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON \ 80 -DEXECUTORCH_BUILD_EXTENSION_MODULE=ON \ 81 -DEXECUTORCH_BUILD_EXTENSION_RUNNER_UTIL=ON \ 82 -DEXECUTORCH_BUILD_EXTENSION_TENSOR=ON \ 83 -Bcmake-android-out 84 85cmake --build cmake-android-out -j16 --target install 86``` 87 88When we set `EXECUTORCH_BUILD_XNNPACK=ON`, we will build the target [`xnnpack_backend`](https://github.com/pytorch/executorch/blob/main/backends/xnnpack/CMakeLists.txt) which in turn is linked into libexecutorch_jni via [CMake](https://github.com/pytorch/executorch/blob/main/examples/demo-apps/android/jni/CMakeLists.txt). 89 902. Build the Android extension: 91 92```bash 93 94# Build the android extension 95cmake extension/android \ 96 -DCMAKE_TOOLCHAIN_FILE="${ANDROID_NDK}"/build/cmake/android.toolchain.cmake \ 97 -DANDROID_ABI="${ANDROID_ABI}" \ 98 -DCMAKE_INSTALL_PREFIX=cmake-android-out \ 99 -Bcmake-android-out/extension/android 100 101cmake --build cmake-android-out/extension/android -j16 102``` 103 104`libexecutorch_jni.so` wraps up the required XNNPACK Backend runtime library from `xnnpack_backend`, and adds an additional JNI layer using fbjni. This is later exposed to Java app. 105 106#### Qualcomm Hexagon NPU 107 1081. Build the CMake target for the library with Qualcomm Hexagon NPU (HTP) backend (XNNPACK also included): 109 110```bash 111export ANDROID_NDK=<path-to-android-ndk> 112export ANDROID_ABI=arm64-v8a 113export QNN_SDK_ROOT=<path-to-qnn-sdk> 114 115rm -rf cmake-android-out && mkdir cmake-android-out && cd cmake-android-out 116cmake . -DCMAKE_INSTALL_PREFIX=cmake-android-out \ 117 -DCMAKE_TOOLCHAIN_FILE="${ANDROID_NDK}/build/cmake/android.toolchain.cmake" \ 118 -DANDROID_ABI="${ANDROID_ABI}" \ 119 -DEXECUTORCH_BUILD_XNNPACK=ON \ 120 -DEXECUTORCH_BUILD_QNN=ON \ 121 -DQNN_SDK_ROOT="${QNN_SDK_ROOT}" \ 122 -DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON \ 123 -DEXECUTORCH_BUILD_EXTENSION_MODULE=ON \ 124 -DEXECUTORCH_BUILD_EXTENSION_RUNNER_UTIL=ON \ 125 -DEXECUTORCH_BUILD_EXTENSION_TENSOR=ON \ 126 -Bcmake-android-out 127 128cmake --build cmake-android-out -j16 --target install 129``` 130Similar to the XNNPACK library, with this setup, we compile `libexecutorch_jni.so` but it adds an additional static library `qnn_executorch_backend` which wraps up Qualcomm HTP runtime library and registers the Qualcomm HTP backend. This is later exposed to Java app. 131 132`qnn_executorch_backend` is built when we turn on CMake option `EXECUTORCH_BUILD_QNN`. It will include the [CMakeLists.txt](https://github.com/pytorch/executorch/blob/main/backends/qualcomm/CMakeLists.txt) from backends/qualcomm where we `add_library(qnn_executorch_backend STATIC)`. 133 1342. Build the Android extension: 135 136```bash 137cmake extension/android \ 138 -DCMAKE_TOOLCHAIN_FILE="${ANDROID_NDK}"/build/cmake/android.toolchain.cmake \ 139 -DANDROID_ABI="${ANDROID_ABI}" \ 140 -DCMAKE_INSTALL_PREFIX=cmake-android-out \ 141 -Bcmake-android-out/extension/android 142 143cmake --build cmake-android-out/extension/android -j16 144``` 145 146## Deploying on Device via Demo App 147 148### Steps for Deploying Model via XNNPACK 149 150```bash 151mkdir -p examples/demo-apps/android/ExecuTorchDemo/app/src/main/jniLibs/arm64-v8a 152cp cmake-android-out/extension/android/libexecutorch_jni.so \ 153 examples/demo-apps/android/ExecuTorchDemo/app/src/main/jniLibs/arm64-v8a/libexecutorch.so 154``` 155 156This allows the Android app to load ExecuTorch runtime with XNNPACK backend as a JNI library. Later, this shared library will be loaded by `NativePeer.java` in Java code. 157 158### Steps for Deploying Model via Qualcomm's AI Engine Direct 159 160```bash 161mkdir -p ../examples/demo-apps/android/ExecuTorchDemo/app/src/main/jniLibs/arm64-v8a 162``` 163 164We need to push some additional Qualcomm HTP backend libraries to the app. Please refer to [Qualcomm docs](build-run-qualcomm-ai-engine-direct-backend.md) here. 165 166```bash 167cp ${QNN_SDK_ROOT}/lib/aarch64-android/libQnnHtp.so ${QNN_SDK_ROOT}/lib/hexagon-v69/unsigned/libQnnHtpV69Skel.so ${QNN_SDK_ROOT}/lib/aarch64-android/libQnnHtpV69Stub.so ${QNN_SDK_ROOT}/lib/aarch64-android/libQnnSystem.so \ 168 examples/demo-apps/android/ExecuTorchDemo/app/src/main/jniLibs/arm64-v8a 169``` 170 171Copy the core libraries: 172 173```bash 174cp cmake-android-out/extension/android/libexecutorch_jni.so \ 175 examples/demo-apps/android/ExecuTorchDemo/app/src/main/jniLibs/arm64-v8a/libexecutorch.so 176cp cmake-android-out/lib/libqnn_executorch_backend.so \ 177 examples/demo-apps/android/ExecuTorchDemo/app/src/main/jniLibs/arm64-v8a/libqnn_executorch_backend.so 178``` 179 180## Running the App 181 1821. Open the project `examples/demo-apps/android/ExecuTorchDemo` with Android Studio. 183 1842. [Run](https://developer.android.com/studio/run) the app (^R). 185 186<img src="_static/img/android_studio.png" alt="Android Studio View" /><br> 187 188On the phone or emulator, you can try running the model: 189<img src="_static/img/android_demo_run.png" alt="Android Demo" /><br> 190 191## Takeaways 192Through this tutorial we've learnt how to build the ExecuTorch runtime library with XNNPACK (or Qualcomm HTP) backend, and expose it to JNI layer to build the Android app running segmentation model. 193 194## Reporting Issues 195 196If you encountered any bugs or issues following this tutorial please file a bug/issue here on [Github](https://github.com/pytorch/executorch/issues/new). 197
