1# Scripts reference 2 3[TOC] 4 5## Record a profile 6 7### app_profiler.py 8 9`app_profiler.py` is used to record profiling data for Android applications and native executables. 10 11```sh 12# Record an Android application. 13$ ./app_profiler.py -p simpleperf.example.cpp 14 15# Record an Android application with Java code compiled into native instructions. 16$ ./app_profiler.py -p simpleperf.example.cpp --compile_java_code 17 18# Record the launch of an Activity of an Android application. 19$ ./app_profiler.py -p simpleperf.example.cpp -a .SleepActivity 20 21# Record a native process. 22$ ./app_profiler.py -np surfaceflinger 23 24# Record a native process given its pid. 25$ ./app_profiler.py --pid 11324 26 27# Record a command. 28$ ./app_profiler.py -cmd \ 29 "dex2oat --dex-file=/data/local/tmp/app-debug.apk --oat-file=/data/local/tmp/a.oat" 30 31# Record an Android application, and use -r to send custom options to the record command. 32$ ./app_profiler.py -p simpleperf.example.cpp \ 33 -r "-e cpu-clock -g --duration 30" 34 35# Record both on CPU time and off CPU time. 36$ ./app_profiler.py -p simpleperf.example.cpp \ 37 -r "-e task-clock -g -f 1000 --duration 10 --trace-offcpu" 38 39# Save profiling data in a custom file (like perf_custom.data) instead of perf.data. 40$ ./app_profiler.py -p simpleperf.example.cpp -o perf_custom.data 41``` 42 43### Profile from launch of an application 44 45Sometimes we want to profile the launch-time of an application. To support this, we added `--app` in 46the record command. The `--app` option sets the package name of the Android application to profile. 47If the app is not already running, the record command will poll for the app process in a loop with 48an interval of 1ms. So to profile from launch of an application, we can first start the record 49command with `--app`, then start the app. Below is an example. 50 51```sh 52$ ./run_simpleperf_on_device.py record --app simpleperf.example.cpp \ 53 -g --duration 1 -o /data/local/tmp/perf.data 54# Start the app manually or using the `am` command. 55``` 56 57To make it convenient to use, `app_profiler.py` supports using the `-a` option to start an Activity 58after recording has started. 59 60```sh 61$ ./app_profiler.py -p simpleperf.example.cpp -a .MainActivity 62``` 63 64### api_profiler.py 65 66`api_profiler.py` is used to control recording in application code. It does preparation work 67before recording, and collects profiling data files after recording. 68 69[Here](./android_application_profiling.md#control-recording-in-application-code) are the details. 70 71### run_simpleperf_without_usb_connection.py 72 73`run_simpleperf_without_usb_connection.py` records profiling data while the USB cable isn't 74connected. Maybe `api_profiler.py` is more suitable, which also don't need USB cable when recording. 75Below is an example. 76 77```sh 78$ ./run_simpleperf_without_usb_connection.py start -p simpleperf.example.cpp 79# After the command finishes successfully, unplug the USB cable, run the 80# SimpleperfExampleCpp app. After a few seconds, plug in the USB cable. 81$ ./run_simpleperf_without_usb_connection.py stop 82# It may take a while to stop recording. After that, the profiling data is collected in perf.data 83# on host. 84``` 85 86### binary_cache_builder.py 87 88The `binary_cache` directory is a directory holding binaries needed by a profiling data file. The 89binaries are expected to be unstripped, having debug information and symbol tables. The 90`binary_cache` directory is used by report scripts to read symbols of binaries. It is also used by 91`report_html.py` to generate annotated source code and disassembly. 92 93By default, `app_profiler.py` builds the binary_cache directory after recording. But we can also 94build `binary_cache` for existing profiling data files using `binary_cache_builder.py`. It is useful 95when you record profiling data using `simpleperf record` directly, to do system wide profiling or 96record without the USB cable connected. 97 98`binary_cache_builder.py` can either pull binaries from an Android device, or find binaries in 99directories on the host (via `-lib`). 100 101```sh 102# Generate binary_cache for perf.data, by pulling binaries from the device. 103$ ./binary_cache_builder.py 104 105# Generate binary_cache, by pulling binaries from the device and finding binaries in 106# SimpleperfExampleCpp. 107$ ./binary_cache_builder.py -lib path_of_SimpleperfExampleCpp 108``` 109 110### run_simpleperf_on_device.py 111 112This script pushes the `simpleperf` executable on the device, and run a simpleperf command on the 113device. It is more convenient than running adb commands manually. 114 115## Viewing the profile 116 117Scripts in this section are for viewing the profile or converting profile data into formats used by 118external UIs. For recommended UIs, see [view_the_profile.md](view_the_profile.md). 119 120### report.py 121 122report.py is a wrapper of the `report` command on the host. It accepts all options of the `report` 123command. 124 125```sh 126# Report call graph 127$ ./report.py -g 128 129# Report call graph in a GUI window implemented by Python Tk. 130$ ./report.py -g --gui 131``` 132 133### report_html.py 134 135`report_html.py` generates `report.html` based on the profiling data. Then the `report.html` can show 136the profiling result without depending on other files. So it can be shown in local browsers or 137passed to other machines. Depending on which command-line options are used, the content of the 138`report.html` can include: chart statistics, sample table, flamegraphs, annotated source code for 139each function, annotated disassembly for each function. 140 141```sh 142# Generate chart statistics, sample table and flamegraphs, based on perf.data. 143$ ./report_html.py 144 145# Add source code. 146$ ./report_html.py --add_source_code --source_dirs path_of_SimpleperfExampleCpp 147 148# Add disassembly. 149$ ./report_html.py --add_disassembly 150 151# Adding disassembly for all binaries can cost a lot of time. So we can choose to only add 152# disassembly for selected binaries. 153$ ./report_html.py --add_disassembly --binary_filter libgame.so 154# Add disassembly and source code for binaries belonging to an app with package name 155# com.example.myapp. 156$ ./report_html.py --add_source_code --add_disassembly --binary_filter com.example.myapp 157 158# report_html.py accepts more than one recording data file. 159$ ./report_html.py -i perf1.data perf2.data 160``` 161 162Below is an example of generating html profiling results for SimpleperfExampleCpp. 163 164```sh 165$ ./app_profiler.py -p simpleperf.example.cpp 166$ ./report_html.py --add_source_code --source_dirs path_of_SimpleperfExampleCpp \ 167 --add_disassembly 168``` 169 170After opening the generated [`report.html`](./report_html.html) in a browser, there are several tabs: 171 172The first tab is "Chart Statistics". You can click the pie chart to show the time consumed by each 173process, thread, library and function. 174 175The second tab is "Sample Table". It shows the time taken by each function. By clicking one row in 176the table, we can jump to a new tab called "Function". 177 178The third tab is "Flamegraph". It shows the graphs generated by [`inferno`](./inferno.md). 179 180The fourth tab is "Function". It only appears when users click a row in the "Sample Table" tab. 181It shows information of a function, including: 182 1831. A flamegraph showing functions called by that function. 1842. A flamegraph showing functions calling that function. 1853. Annotated source code of that function. It only appears when there are source code files for 186 that function. 1874. Annotated disassembly of that function. It only appears when there are binaries containing that 188 function. 189 190### inferno 191 192[`inferno`](./inferno.md) is a tool used to generate flamegraph in a html file. 193 194```sh 195# Generate flamegraph based on perf.data. 196# On Windows, use inferno.bat instead of ./inferno.sh. 197$ ./inferno.sh -sc --record_file perf.data 198 199# Record a native program and generate flamegraph. 200$ ./inferno.sh -np surfaceflinger 201``` 202 203### purgatorio 204 205[`purgatorio`](../scripts/purgatorio/README.md) is a visualization tool to show samples in time order. 206 207### pprof_proto_generator.py 208 209It converts a profiling data file into `pprof.proto`, a format used by [pprof](https://github.com/google/pprof). 210 211```sh 212# Convert perf.data in the current directory to pprof.proto format. 213$ ./pprof_proto_generator.py 214# Show report in pdf format. 215$ pprof -pdf pprof.profile 216 217# Show report in html format. To show disassembly, add --tools option like: 218# --tools=objdump:<ndk_path>/toolchains/llvm/prebuilt/linux-x86_64/aarch64-linux-android/bin 219# To show annotated source or disassembly, select `top` in the view menu, click a function and 220# select `source` or `disassemble` in the view menu. 221$ pprof -http=:8080 pprof.profile 222``` 223 224### gecko_profile_generator.py 225 226Converts `perf.data` to [Gecko Profile 227Format](https://github.com/firefox-devtools/profiler/blob/main/docs-developer/gecko-profile-format.md), 228a format readable by both the [Perfetto UI](https://ui.perfetto.dev/) and 229[Firefox Profiler](https://profiler.firefox.com/). 230[View the profile](view_the_profile.md) provides more information on both options. 231 232Usage: 233 234``` 235# Record a profile of your application 236$ ./app_profiler.py -p simpleperf.example.cpp 237 238# Convert and gzip. 239$ ./gecko_profile_generator.py -i perf.data | gzip > gecko-profile.json.gz 240``` 241 242Then open `gecko-profile.json.gz` in https://ui.perfetto.dev/ or 243https://profiler.firefox.com/. 244 245### report_sample.py 246 247`report_sample.py` converts a profiling data file into the `perf script` text format output by 248`linux-perf-tool`. 249 250This format can be imported into: 251 252- [Perfetto](https://ui.perfetto.dev) 253- [FlameGraph](https://github.com/brendangregg/FlameGraph) 254- [Flamescope](https://github.com/Netflix/flamescope) 255- [Firefox 256 Profiler](https://github.com/firefox-devtools/profiler/blob/main/docs-user/guide-perf-profiling.md), 257 but prefer using `gecko_profile_generator.py`. 258- [Speedscope](https://github.com/jlfwong/speedscope/wiki/Importing-from-perf-(linux)) 259 260```sh 261# Record a profile to perf.data 262$ ./app_profiler.py <args> 263 264# Convert perf.data in the current directory to a format used by FlameGraph. 265$ ./report_sample.py --symfs binary_cache >out.perf 266 267$ git clone https://github.com/brendangregg/FlameGraph.git 268$ FlameGraph/stackcollapse-perf.pl out.perf >out.folded 269$ FlameGraph/flamegraph.pl out.folded >a.svg 270``` 271 272### stackcollapse.py 273 274`stackcollapse.py` converts a profiling data file (`perf.data`) to [Brendan 275Gregg's "Folded Stacks" 276format](https://queue.acm.org/detail.cfm?id=2927301#:~:text=The%20folded%20stack%2Dtrace%20format,trace%2C%20followed%20by%20a%20semicolon). 277 278Folded Stacks are lines of semicolon-delimited stack frames, root to leaf, 279followed by a count of events sampled in that stack, e.g.: 280 281``` 282BusyThread;__start_thread;__pthread_start(void*);java.lang.Thread.run 17889729 283``` 284 285All similar stacks are aggregated and sample timestamps are unused. 286 287Folded Stacks format is readable by: 288 289- The [FlameGraph](https://github.com/brendangregg/FlameGraph) toolkit 290- [Inferno](https://github.com/jonhoo/inferno) (Rust port of FlameGraph) 291- [Speedscope](https://speedscope.app/) 292 293Example: 294 295```sh 296# Record a profile to perf.data 297$ ./app_profiler.py <args> 298 299# Convert to Folded Stacks format 300$ ./stackcollapse.py --kernel --jit | gzip > profile.folded.gz 301 302# Visualise with FlameGraph with Java Stacks and nanosecond times 303$ git clone https://github.com/brendangregg/FlameGraph.git 304$ gunzip -c profile.folded.gz \ 305 | FlameGraph/flamegraph.pl --color=java --countname=ns \ 306 > profile.svg 307``` 308 309## simpleperf_report_lib.py 310 311`simpleperf_report_lib.py` is a Python library used to parse profiling data files generated by the 312record command. Internally, it uses libsimpleperf_report.so to do the work. Generally, for each 313profiling data file, we create an instance of ReportLib, pass it the file path (via SetRecordFile). 314Then we can read all samples through GetNextSample(). For each sample, we can read its event info 315(via GetEventOfCurrentSample), symbol info (via GetSymbolOfCurrentSample) and call chain info 316(via GetCallChainOfCurrentSample). We can also get some global information, like record options 317(via GetRecordCmd), the arch of the device (via GetArch) and meta strings (via MetaInfo). 318 319Examples of using `simpleperf_report_lib.py` are in `report_sample.py`, `report_html.py`, 320`pprof_proto_generator.py` and `inferno/inferno.py`. 321 322## ipc.py 323`ipc.py`captures the instructions per cycle (IPC) of the system during a specified duration. 324 325Example: 326```sh 327./ipc.py 328./ipc.py 2 20 # Set interval to 2 secs and total duration to 20 secs 329./ipc.py -p 284 -C 4 # Only profile the PID 284 while running on core 4 330./ipc.py -c 'sleep 5' # Only profile the command to run 331``` 332 333The results look like: 334``` 335K_CYCLES K_INSTR IPC 33636840 14138 0.38 33770701 27743 0.39 338104562 41350 0.40 339138264 54916 0.40 340``` 341 342## sample_filter.py 343 344`sample_filter.py` generates sample filter files as documented in [sample_filter.md](https://android.googlesource.com/platform/system/extras/+/refs/heads/main/simpleperf/doc/sample_filter.md). 345A filter file can be passed in `--filter-file` when running report scripts. 346 347For example, it can be used to split a large recording file into several report files. 348 349```sh 350$ sample_filter.py -i perf.data --split-time-range 2 -o sample_filter 351$ gecko_profile_generator.py -i perf.data --filter-file sample_filter_part1 \ 352 | gzip >profile-part1.json.gz 353$ gecko_profile_generator.py -i perf.data --filter-file sample_filter_part2 \ 354 | gzip >profile-part2.json.gz 355``` 356
