1# ExecuTorch in Portable Mode 2 3This dir contains demos to illustrate an end-to-end workflow of using ExecuTorch in [portable mode](../../docs/source/concepts.md#portable-mode-lean-mode). 4 5 6## Directory structure 7```bash 8examples/portable 9├── scripts # Python scripts to illustrate export workflow 10│ ├── export.py 11│ └── export_and_delegate.py 12├── custom_ops # Contains examples to register custom operators into PyTorch as well as register its kernels into ExecuTorch runtime 13├── executor_runner # Contains an example C++ wrapper around the ExecuTorch runtime 14└── README.md # This file 15``` 16 17## Using portable mode 18 19We will walk through an example model to generate a `.pte` file in [portable mode](../../docs/source/concepts.md#portable-mode-lean-mode) from a python `torch.nn.module` 20from the [`models/`](../models) directory using scripts in the `portable/scripts` directory. Then we will run on the `.pte` model on the ExecuTorch runtime. For that we will use `executor_runner`. 21 22 231. Following the setup guide in [Setting up ExecuTorch](https://pytorch.org/executorch/stable/getting-started-setup) 24you should be able to get the basic development environment for ExecuTorch working. 25 262. Using the script `portable/scripts/export.py` generate a model binary file by selecting a 27model name from the list of available models in the `models` dir. 28 29 30```bash 31cd executorch # To the top level dir 32 33# To get a list of example models 34python3 -m examples.portable.scripts.export -h 35 36# To generate a specific pte model 37python3 -m examples.portable.scripts.export --model_name="mv2" # for MobileNetv2 38 39# This should generate ./mv2.pte file, if successful. 40``` 41 42Use `-h` (or `--help`) to see all the supported models. 43 443. Once we have the model binary (`.pte`) file, then let's run it with ExecuTorch runtime using the `executor_runner`. 45 46```bash 47# Build the tool from the top-level `executorch` directory. 48(rm -rf cmake-out \ 49 && mkdir cmake-out \ 50 && cd cmake-out \ 51 && cmake -DEXECUTORCH_PAL_DEFAULT=posix ..) \ 52 && cmake --build cmake-out -j32 --target executor_runner 53 54# Run the tool on the generated model. 55./cmake-out/executor_runner --model_path mv2.pte 56``` 57 58This will run the model with all input tensor elements set to `1`, and print 59the outputs. For example: 60``` 61I 00:00:00.004885 executorch:executor_runner.cpp:73] Model file mv2.pte is loaded. 62I 00:00:00.004902 executorch:executor_runner.cpp:82] Using method forward 63I 00:00:00.004906 executorch:executor_runner.cpp:129] Setting up planned buffer 0, size 18652672. 64I 00:00:00.007243 executorch:executor_runner.cpp:152] Method loaded. 65I 00:00:00.007490 executorch:executor_runner.cpp:162] Inputs prepared. 66I 00:00:06.887939 executorch:executor_runner.cpp:171] Model executed successfully. 67I 00:00:06.887975 executorch:executor_runner.cpp:175] 1 outputs: 68Output 0: tensor(sizes=[1, 1000], [ 69 -0.50986, 0.300638, 0.0953877, 0.147722, 0.231202, 0.338555, 0.20689, -0.057578, -0.389269, -0.060687, 70 -0.0213992, -0.121035, -0.288955, 0.134054, -0.171976, -0.0603627, 0.0203591, -0.0585333, 0.337855, -0.0718644, 71 0.490758, 0.524144, 0.197857, 0.122066, -0.35913, 0.109461, 0.347747, 0.478515, 0.226558, 0.0363523, 72 ..., 73 -0.227163, 0.567008, 0.202894, 0.71008, 0.421649, -0.00655106, 0.0114818, 0.398908, 0.0349851, -0.163213, 74 0.187843, -0.154387, -0.22716, 0.150879, 0.265103, 0.087489, -0.188225, 0.0213046, -0.0293779, -0.27963, 75 0.421221, 0.10045, -0.506771, -0.115818, -0.693015, -0.183256, 0.154783, -0.410679, 0.0119293, 0.449714, 76]) 77``` 78 79## Custom Operator Registration 80 81Explore the demos in the [`custom_ops/`](./custom_ops) directory to learn how to register custom operators into ExecuTorch as well as register its kernels into ExecuTorch runtime. 82