1# Building and Running ExecuTorch with Core ML Backend 2 3Core ML delegate uses Core ML APIs to enable running neural networks via Apple's hardware acceleration. For more about Core ML you can read [here](https://developer.apple.com/documentation/coreml). In this tutorial, we will walk through the steps of lowering a PyTorch model to Core ML delegate 4 5 6::::{grid} 2 7:::{grid-item-card} What you will learn in this tutorial: 8:class-card: card-prerequisites 9* In this tutorial you will learn how to export [MobileNet V3](https://pytorch.org/vision/main/models/mobilenetv3.html) model so that it runs on Core ML backend. 10* You will also learn how to deploy and run the exported model on a supported Apple device. 11::: 12:::{grid-item-card} Tutorials we recommend you complete before this: 13:class-card: card-prerequisites 14* [Introduction to ExecuTorch](intro-how-it-works.md) 15* [Setting up ExecuTorch](getting-started-setup.md) 16* [Building ExecuTorch with CMake](runtime-build-and-cross-compilation.md) 17* [ExecuTorch iOS Demo App](demo-apps-ios.md) 18::: 19:::: 20 21 22## Prerequisites (Hardware and Software) 23 24In order to be able to successfully build and run the ExecuTorch's Core ML backend you'll need the following hardware and software components. 25 26### Hardware: 27- A [mac](https://www.apple.com/mac/) system for building. 28- A [mac](https://www.apple.com/mac/) or [iPhone](https://www.apple.com/iphone/) or [iPad](https://www.apple.com/ipad/) or [Apple TV](https://www.apple.com/tv-home/) device for running the model. 29 30### Software: 31 32- [Xcode](https://developer.apple.com/documentation/xcode) >= 14.1, [macOS](https://developer.apple.com/macos) >= 13.0 for building. 33- [macOS](https://developer.apple.com/macos) >= 13.0, [iOS](https://developer.apple.com/ios/) >= 16.0, [iPadOS](https://developer.apple.com/ipados/) >= 16.0, and [tvOS](https://developer.apple.com/tvos/) >= 16.0 for running the model. 34 35## Setting up your developer environment 36 371. Make sure that you have completed the ExecuTorch setup tutorials linked to at the top of this page and setup the environment. 382. Run `install_requirements.sh` to install dependencies required by the **Core ML** backend. 39 40```bash 41cd executorch 42./backends/apple/coreml/scripts/install_requirements.sh 43``` 443. Install [Xcode](https://developer.apple.com/xcode/). 454. Install Xcode Command Line Tools. 46 47```bash 48xcode-select --install 49``` 50 51## Build 52 53### AOT (Ahead-of-time) components: 54 55 56**Exporting a Core ML delegated Program**: 57- In this step, you will lower the [MobileNet V3](https://pytorch.org/vision/main/models/mobilenetv3.html) model to the Core ML backend and export the ExecuTorch program. You'll then deploy and run the exported program on a supported Apple device using Core ML backend. 58```bash 59cd executorch 60 61# Generates ./mv3_coreml_all.pte file. 62python3 -m examples.apple.coreml.scripts.export --model_name mv3 63``` 64 65- Core ML backend uses [coremltools](https://apple.github.io/coremltools/docs-guides/source/overview-coremltools.html) to lower [Edge dialect](ir-exir.md#edge-dialect) to Core ML format and then bundles it in the `.pte` file. 66 67 68### Runtime: 69 70**Running a Core ML delegated Program**: 711. Build the runner. 72```bash 73cd executorch 74 75# Builds `coreml_executor_runner`. 76./examples/apple/coreml/scripts/build_executor_runner.sh 77``` 782. Run the CoreML delegated program. 79```bash 80cd executorch 81 82# Runs the exported mv3 model using the Core ML backend. 83./coreml_executor_runner --model_path mv3_coreml_all.pte 84``` 85 86**Profiling a Core ML delegated Program**: 87 88Note that profiling is supported on [macOS](https://developer.apple.com/macos) >= 14.4. 89 901. [Optional] Generate an [ETRecord](./etrecord.rst) when exporting your model. 91```bash 92cd executorch 93 94# Generates `mv3_coreml_all.pte` and `mv3_coreml_etrecord.bin` files. 95python3 -m examples.apple.coreml.scripts.export --model_name mv3 --generate_etrecord 96``` 97 982. Build the runner. 99```bash 100# Builds `coreml_executor_runner`. 101./examples/apple/coreml/scripts/build_executor_runner.sh 102``` 1033. Run and generate an [ETDump](./etdump.md). 104```bash 105cd executorch 106 107# Generate the ETDump file. 108./coreml_executor_runner --model_path mv3_coreml_all.pte --profile_model --etdump_path etdump.etdp 109``` 110 1114. Create an instance of the [Inspector API](./model-inspector.rst) by passing in the [ETDump](./etdump.md) you have sourced from the runtime along with the optionally generated [ETRecord](./etrecord.rst) from step 1 or execute the following command in your terminal to display the profiling data table. 112```bash 113python examples/apple/coreml/scripts/inspector_cli.py --etdump_path etdump.etdp --etrecord_path mv3_coreml.bin 114``` 115 116 117## Deploying and running on a device 118 119**Running the Core ML delegated Program in the Demo iOS App**: 1201. Please follow the [Export Model](demo-apps-ios.md#models-and-labels) step of the tutorial to bundle the exported [MobileNet V3](https://pytorch.org/vision/main/models/mobilenetv3.html) program. You only need to do the Core ML part. 121 1222. Complete the [Build Runtime and Backends](demo-apps-ios.md#build-runtime-and-backends) section of the tutorial. When building the frameworks you only need the `coreml` option. 123 1243. Complete the [Final Steps](demo-apps-ios.md#final-steps) section of the tutorial to build and run the demo app. 125 126<br>**Running the Core ML delegated Program in your App** 1271. Build frameworks, running the following will create a `executorch.xcframework` and `coreml_backend.xcframework` in the `cmake-out` directory. 128```bash 129cd executorch 130./build/build_apple_frameworks.sh --coreml 131``` 1322. Create a new [Xcode project](https://developer.apple.com/documentation/xcode/creating-an-xcode-project-for-an-app#) or open an existing project. 133 1343. Drag the `executorch.xcframework` and `coreml_backend.xcframework` generated from Step 2 to Frameworks. 135 1364. Go to the project's [Build Phases](https://developer.apple.com/documentation/xcode/customizing-the-build-phases-of-a-target) - Link Binaries With Libraries, click the + sign, and add the following frameworks: 137``` 138executorch.xcframework 139coreml_backend.xcframework 140Accelerate.framework 141CoreML.framework 142libsqlite3.tbd 143``` 1445. Add the exported program to the [Copy Bundle Phase](https://developer.apple.com/documentation/xcode/customizing-the-build-phases-of-a-target#Copy-files-to-the-finished-product) of your Xcode target. 145 1466. Please follow the [Runtime APIs Tutorial](extension-module.md) to integrate the code for loading an ExecuTorch program. 147 1487. Update the code to load the program from the Application's bundle. 149``` objective-c 150NSURL *model_url = [NBundle.mainBundle URLForResource:@"mv3_coreml_all" extension:@"pte"]; 151 152Result<executorch::extension::FileDataLoader> loader = 153 executorch::extension::FileDataLoader::from(model_url.path.UTF8String); 154``` 155 1568. Use [Xcode](https://developer.apple.com/documentation/xcode/building-and-running-an-app#Build-run-and-debug-your-app) to deploy the application on the device. 157 1589. The application can now run the [MobileNet V3](https://pytorch.org/vision/main/models/mobilenetv3.html) model on the Core ML backend. 159 160<br>In this tutorial, you have learned how to lower the [MobileNet V3](https://pytorch.org/vision/main/models/mobilenetv3.html) model to the Core ML backend, deploy, and run it on an Apple device. 161 162## Frequently encountered errors and resolution. 163 164If you encountered any bugs or issues following this tutorial please file a bug/issue [here](https://github.com/pytorch/executorch/issues) with tag #coreml. 165