1# Chromium's Java Toolchain 2 3This doc aims to describe the Chrome build process that takes a set of `.java` 4files and turns them into a `classes.dex` file. 5 6[TOC] 7 8## Core GN Target Types 9 10The following have `supports_android` and `requires_android` set to false by 11default: 12* `java_library()`: Compiles `.java` -> `.jar` 13* `java_prebuilt()`: Imports a prebuilt `.jar` file. 14 15The following have `supports_android` and `requires_android` set to true. They 16also have a default `jar_excluded_patterns` set (more on that later): 17* `android_library()` 18* `android_java_prebuilt()` 19 20All target names must end with "_java" so that the build system can distinguish 21them from non-java targets (or [other variations](https://cs.chromium.org/chromium/src/build/config/android/internal_rules.gni?rcl=ec2c17d7b4e424e060c3c7972842af87343526a1&l=20)). 22 23Most targets produce two separate `.jar` files: 24* Device `.jar`: Used to produce `.dex.jar`, which is used on-device. 25* Host `.jar`: For use on the host machine (`junit_binary` / `java_binary`). 26 * Host `.jar` files live in `lib.java/` so that they are archived in 27 builder/tester bots (which do not archive `obj/`). 28 29## From Source to Final Dex 30 31### Step 1: Create interface .jar with turbine or ijar 32 33What are interface jars?: 34 35* They contain `.class` files with all private symbols and all method bodies 36 removed. 37* Dependant targets use interface `.jar` files to skip having to be rebuilt 38 when only private implementation details change. 39 40For prebuilt `.jar` files: we use [//third_party/ijar] to create interface 41`.jar` files from the prebuilt ones. 42 43For non-prebuilt `.jar` files`: we use [//third_party/turbine] to create 44interface `.jar` files directly from `.java` source files. Turbine is faster 45than javac because it does not compile method bodies. Although Turbine causes 46us to compile files twice, it speeds up builds by allowing `javac` compilation 47of targets to happen concurrently with their dependencies. We also use Turbine 48to run our annotation processors. 49 50[//third_party/ijar]: /third_party/ijar/README.chromium 51[//third_party/turbine]: /third_party/turbine/README.chromium 52 53### Step 2a: Compile with javac 54 55This step is the only step that does not apply to prebuilt targets. 56 57* All `.java` files in a target are compiled by `javac` into `.class` files. 58 * This includes `.java` files that live within `.srcjar` files, referenced 59 through `srcjar_deps`. 60* The `classpath` used when compiling a target is comprised of `.jar` files of 61 its deps. 62 * When deps are library targets, the Step 1 `.jar` file is used. 63 * When deps are prebuilt targets, the original `.jar` file is used. 64 * All `.jar` processing done in subsequent steps does not impact compilation 65 classpath. 66* `.class` files are zipped into an output `.jar` file. 67* There is **no support** for incremental compilation at this level. 68 * If one source file changes within a library, then the entire library is 69 recompiled. 70 * Prefer smaller targets to avoid slow compiles. 71 72### Step 2b: Compile with ErrorProne 73 74This step can be disabled via GN arg: `use_errorprone_java_compiler = false` 75 76* Concurrently with step 1a: [ErrorProne] compiles java files and checks for bug 77 patterns, including some [custom to Chromium][ep_plugins]. 78* ErrorProne used to replace step 1a, but was changed to a concurrent step after 79 being identified as being slower. 80 81[ErrorProne]: https://errorprone.info/ 82[ep_plugins]: /tools/android/errorprone_plugin/ 83 84### Step 3: Desugaring (Device .jar Only) 85 86This step happens only when targets have `supports_android = true`. It is not 87applied to `.jar` files used by `junit_binary`. 88 89* `//third_party/bazel/desugar` converts certain Java 8 constructs, such as 90 lambdas and default interface methods, into constructs that are compatible 91 with Java 7. 92 93### Step 4: Instrumenting (Device .jar Only) 94 95This step happens only when this GN arg is set: `use_jacoco_coverage = true` 96 97* [Jacoco] adds instrumentation hooks to methods. 98 99[Jacoco]: https://www.eclemma.org/jacoco/ 100 101### Step 5: Filtering 102 103This step happens only when targets that have `jar_excluded_patterns` or 104`jar_included_patterns` set (e.g. all `android_` targets). 105 106* Remove `.class` files that match the filters from the `.jar`. These `.class` 107 files are generally those that are re-created with different implementations 108 further on in the build process. 109 * E.g.: `R.class` files - a part of [Android Resources]. 110 * E.g.: `GEN_JNI.class` - a part of our [JNI] glue. 111 * E.g.: `AppHooksImpl.class` - how `chrome_java` wires up different 112 implementations for [non-public builds][apphooks]. 113 114[JNI]: /third_party/jni_zero/README.md 115[Android Resources]: life_of_a_resource.md 116[apphooks]: /chrome/android/java/src/org/chromium/chrome/browser/AppHooksImpl.java 117 118### Step 6: Per-Library Dexing 119 120This step happens only when targets have `supports_android = true`. 121 122* [d8] converts `.jar` files containing `.class` files into `.dex.jar` files 123 containing `classes.dex` files. 124* Dexing is incremental - it will reuse dex'ed classes from a previous build if 125 the corresponding `.class` file is unchanged. 126* These per-library `.dex.jar` files are used directly by [incremental install], 127 and are inputs to the Apk step when `enable_proguard = false`. 128 * Even when `is_java_debug = false`, many apk targets do not enable ProGuard 129 (e.g. unit tests). 130 131[d8]: https://developer.android.com/studio/command-line/d8 132[incremental install]: /build/android/incremental_install/README.md 133 134### Step 7: Apk / Bundle Module Compile 135 136* Each `android_apk` and `android_bundle_module` template has a nested 137 `java_library` target. The nested library includes final copies of files 138 stripped out by prior filtering steps. These files include: 139 * Final `R.java` files, created by `compile_resources.py`. 140 * Final `GEN_JNI.java` for [JNI glue]. 141 * `BuildConfig.java` and `NativeLibraries.java` (//base dependencies). 142 143[JNI glue]: /third_party/jni_zero/README.md 144 145### Step 8: Final Dexing 146 147This step is skipped when building using [Incremental Install]. 148 149When `is_java_debug = true`: 150* [d8] merges all library `.dex.jar` files into a final `.mergeddex.jar`. 151 152When `is_java_debug = false`: 153* [R8] performs whole-program optimization on all library `lib.java` `.jar` 154 files and outputs a final `.r8dex.jar`. 155 * For App Bundles, R8 creates a `.r8dex.jar` for each module. 156 157[Incremental Install]: /build/android/incremental_install/README.md 158[R8]: https://r8.googlesource.com/r8 159 160## Test APKs with apk_under_test 161 162Test APKs are normal APKs that contain an `<instrumentation>` tag within their 163`AndroidManifest.xml`. If this tag specifies an `android:targetPackage` 164different from itself, then Android will add that package's `classes.dex` to the 165test APK's Java classpath when run. In GN, you can enable this behavior using 166the `apk_under_test` parameter on `instrumentation_test_apk` targets. Using it 167is discouraged if APKs have `proguard_enabled=true`. 168 169### Difference in Final Dex 170 171When `enable_proguard=false`: 172* Any library depended on by the test APK that is also depended on by the 173 apk-under-test is excluded from the test APK's final dex step. 174 175When `enable_proguard=true`: 176* Test APKs cannot make use of the apk-under-test's dex because only symbols 177 explicitly kept by `-keep` directives are guaranteed to exist after 178 ProGuarding. As a work-around, test APKs include all of the apk-under-test's 179 libraries directly in its own final dex such that the under-test apk's Java 180 code is never used (because it is entirely shadowed by the test apk's dex). 181 * We've found this configuration to be fragile, and are trying to [move away 182 from it](https://bugs.chromium.org/p/chromium/issues/detail?id=890452). 183 184### Difference in GEN_JNI.java 185* Calling native methods using [JNI glue] requires that a `GEN_JNI.java` class 186 be generated that contains all native methods for an APK. There cannot be 187 conflicting `GEN_JNI` classes in both the test apk and the apk-under-test, so 188 only the apk-under-test has one generated for it. As a result this, 189 instrumentation test APKs that use apk-under-test cannot use native methods 190 that aren't already part of the apk-under-test. 191 192## How to Generate Java Source Code 193There are two ways to go about generating source files: Annotation Processors 194and custom build steps. 195 196### Annotation Processors 197* These are run by `javac` as part of the compile step. 198* They **cannot** modify the source files that they apply to. They can only 199 generate new sources. 200* Use these when: 201 * an existing Annotation Processor does what you want 202 (E.g. Dagger, AutoService, etc.), or 203 * you need to understand Java types to do generation. 204 205### Custom Build Steps 206* These use discrete build actions to generate source files. 207 * Some generate `.java` directly, but most generate a zip file of sources 208 (called a `.srcjar`) to simplify the number of inputs / outputs. 209* Examples of existing templates: 210 * `jinja_template`: Generates source files using [Jinja]. 211 * `java_cpp_template`: Generates source files using the C preprocessor. 212 * `java_cpp_enum`: Generates `@IntDef`s based on enums within `.h` files. 213 * `java_cpp_strings`: Generates String constants based on strings defined in 214 `.cc` files. 215* Custom build steps are preferred over Annotation Processors because they are 216 generally easier to understand, and can run in parallel with other steps 217 (rather than being tied to compiles). 218 219[Jinja]: https://palletsprojects.com/p/jinja/ 220 221## Static Analysis & Code Checks 222 223See [static_analysis.md](static_analysis.md) 224