1#!/usr/bin/env python3 2# 3# Copyright 2014 The Chromium Authors 4# Use of this source code is governed by a BSD-style license that can be 5# found in the LICENSE file. 6 7"""Writes a build_config file. 8 9The build_config file for a target is a json file containing information about 10how to build that target based on the target's dependencies. This includes 11things like: the javac classpath, the list of android resources dependencies, 12etc. It also includes the information needed to create the build_config for 13other targets that depend on that one. 14 15Android build scripts should not refer to the build_config directly, and the 16build specification should instead pass information in using the special 17file-arg syntax (see build_utils.py:ExpandFileArgs). That syntax allows passing 18of values in a json dict in a file and looks like this: 19 --python-arg=@FileArg(build_config_path:javac:classpath) 20 21Note: If paths to input files are passed in this way, it is important that: 22 1. inputs/deps of the action ensure that the files are available the first 23 time the action runs. 24 2. Either (a) or (b) 25 a. inputs/deps ensure that the action runs whenever one of the files changes 26 b. the files are added to the action's depfile 27 28NOTE: All paths within .build_config files are relative to $OUTPUT_CHROMIUM_DIR. 29 30This is a technical note describing the format of .build_config files. 31Please keep it updated when changing this script. For extraction and 32visualization instructions, see build/android/docs/build_config.md 33 34------------- BEGIN_MARKDOWN --------------------------------------------------- 35The .build_config file format 36=== 37 38# Introduction 39 40This document tries to explain the format of `.build_config` generated during 41the Android build of Chromium. For a higher-level explanation of these files, 42please read 43[build/android/docs/build_config.md](build/android/docs/build_config.md). 44 45# The `deps_info` top-level dictionary: 46 47All `.build_config` files have a required `'deps_info'` key, whose value is a 48dictionary describing the target and its dependencies. The latter has the 49following required keys: 50 51## Required keys in `deps_info`: 52 53* `deps_info['type']`: The target type as a string. 54 55 The following types are known by the internal GN build rules and the 56 build scripts altogether: 57 58 * [java_binary](#target_java_binary) 59 * [java_annotation_processor](#target_java_annotation_processor) 60 * [robolectric_binary](#target_robolectric_binary) 61 * [java_library](#target_java_library) 62 * [android_assets](#target_android_assets) 63 * [android_resources](#target_android_resources) 64 * [android_apk](#target_android_apk) 65 * [android_app_bundle_module](#target_android_app_bundle_module) 66 * [android_app_bundle](#target_android_app_bundle) 67 * [dist_jar](#target_dist_jar) 68 * [dist_aar](#target_dist_aar) 69 * [group](#target_group) 70 71 See later sections for more details of some of these. 72 73* `deps_info['path']`: Path to the target's `.build_config` file. 74 75* `deps_info['name']`: Nothing more than the basename of `deps_info['path']` 76at the moment. 77 78* `deps_info['deps_configs']`: List of paths to the `.build_config` files of 79all *direct* dependencies of the current target. 80 81 NOTE: Because the `.build_config` of a given target is always generated 82 after the `.build_config` of its dependencies, the `write_build_config.py` 83 script can use chains of `deps_configs` to compute transitive dependencies 84 for each target when needed. 85 86## Optional keys in `deps_info`: 87 88The following keys will only appear in the `.build_config` files of certain 89target types: 90 91* `deps_info['requires_android']`: True to indicate that the corresponding 92code uses Android-specific APIs, and thus cannot run on the host within a 93regular JVM. May only appear in Java-related targets. 94 95* `deps_info['supports_android']`: 96May appear in Java-related targets, and indicates that 97the corresponding code doesn't use Java APIs that are not available on 98Android. As such it may run either on the host or on an Android device. 99 100* `deps_info['assets']`: 101Only seen for the [`android_assets`](#target_android_assets) type. See below. 102 103* `deps_info['package_name']`: Java package name associated with this target. 104 105 NOTE: For `android_resources` targets, 106 this is the package name for the corresponding R class. For `android_apk` 107 targets, this is the corresponding package name. This does *not* appear for 108 other target types. 109 110* `deps_info['android_manifest']`: 111Path to an AndroidManifest.xml file related to the current target. 112 113* `deps_info['base_module_config']`: 114Only seen for the [`android_app_bundle`](#target_android_app_bundle) type. 115Path to the base module for the bundle. 116 117* `deps_info['module_name']`: 118Only seen for the 119[`android_app_bundle_module`](#target_android_app_bundle_module) 120type. The name of the feature module. 121 122* `deps_info['dependency_zips']`: 123List of `deps_info['resources_zip']` entries for all `android_resources` 124dependencies for the current target. 125 126* `deps_info['extra_package_names']`: 127Always empty for `android_resources` types. Otherwise, 128the list of `deps_info['package_name']` entries for all `android_resources` 129dependencies for the current target. Computed automatically by 130`write_build_config.py`. 131 132* `deps_info['dependency_r_txt_files']`: 133Exists only on dist_aar. It is the list of deps_info['r_text_path'] from 134transitive dependencies. Computed automatically. 135 136 137# `.build_config` target types description: 138 139## <a name="target_group">Target type `group`</a>: 140 141This type corresponds to a simple target that is only used to group 142dependencies. It matches the `java_group()` GN template. Its only top-level 143`deps_info` keys are `supports_android` (always True), and `deps_configs`. 144 145 146## <a name="target_android_resources">Target type `android_resources`</a>: 147 148This type corresponds to targets that are used to group Android resource files. 149For example, all `android_resources` dependencies of an `android_apk` will 150end up packaged into the final APK by the build system. 151 152It uses the following keys: 153 154 155* `deps_info['res_sources_path']`: 156Path to file containing a list of resource source files used by the 157android_resources target. 158 159* `deps_info['resources_zip']`: 160*Required*. Path to the `.resources.zip` file that contains all raw/uncompiled 161resource files for this target (and also no `R.txt`, `R.java` or `R.class`). 162 163 If `deps_info['res_sources_path']` is missing, this must point to a prebuilt 164 `.aar` archive containing resources. Otherwise, this will point to a zip 165 archive generated at build time, wrapping the sources listed in 166 `deps_info['res_sources_path']` into a single zip file. 167 168* `deps_info['package_name']`: 169Java package name that the R class for this target belongs to. 170 171* `deps_info['android_manifest']`: 172Optional. Path to the top-level Android manifest file associated with these 173resources (if not provided, an empty manifest will be used to generate R.txt). 174 175* `deps_info['resource_overlay']`: 176Optional. Whether the resources in resources_zip should override resources with 177the same name. Does not affect the behaviour of any android_resources() 178dependencies of this target. If a target with resource_overlay=true depends 179on another target with resource_overlay=true the target with the dependency 180overrides the other. 181 182* `deps_info['r_text_path']`: 183Provide the path to the `R.txt` file that describes the resources wrapped by 184this target. Normally this file is generated from the content of the resource 185directories or zip file, but some targets can provide their own `R.txt` file 186if they want. 187 188* `deps_info['srcjar_path']`: 189Path to the `.srcjar` file that contains the auto-generated `R.java` source 190file corresponding to the content of `deps_info['r_text_path']`. This is 191*always* generated from the content of `deps_info['r_text_path']` by the 192`build/android/gyp/process_resources.py` script. 193 194## <a name="target_android_assets">Target type `android_assets`</a>: 195 196This type corresponds to targets used to group Android assets, i.e. liberal 197files that will be placed under `//assets/` within the final APK. 198 199These use an `deps_info['assets']` key to hold a dictionary of values related 200to assets covered by this target. 201 202* `assets['sources']`: 203The list of all asset source paths for this target. Each source path can 204use an optional `:<zipPath>` suffix, where `<zipPath>` is the final location 205of the assets (relative to `//assets/`) within the APK. 206 207* `assets['outputs']`: 208Optional. Some of the sources might be renamed before being stored in the 209final //assets/ sub-directory. When this happens, this contains a list of 210all renamed output file paths 211 212 NOTE: When not empty, the first items of `assets['sources']` must match 213 every item in this list. Extra sources correspond to non-renamed sources. 214 215 NOTE: This comes from the `asset_renaming_destinations` parameter for the 216 `android_assets()` GN template. 217 218* `assets['disable_compression']`: 219Optional. Will be True to indicate that these assets should be stored 220uncompressed in the final APK. For example, this is necessary for locale 221.pak files used by the System WebView feature. 222 223* `assets['treat_as_locale_paks']`: 224Optional. Will be True to indicate that these assets are locale `.pak` files 225(containing localized strings for C++). These are later processed to generate 226a special ``.build_config`.java` source file, listing all supported Locales in 227the current build. 228 229 230## <a name="target_java_library">Target type `java_library`</a>: 231 232This type is used to describe target that wrap Java bytecode, either created 233by compiling sources, or providing them with a prebuilt jar. 234 235* `deps_info['public_deps_configs']`: List of paths to the `.build_config` files 236of *direct* dependencies of the current target which are exposed as part of the 237current target's public API. 238 239* `deps_info['unprocessed_jar_path']`: 240Path to the original .jar file for this target, before any kind of processing 241through Proguard or other tools. For most targets this is generated 242from sources, with a name like `$target_name.javac.jar`. However, when using 243a prebuilt jar, this will point to the source archive directly. 244 245* `deps_info['device_jar_path']`: 246Path to a file that is the result of processing 247`deps_info['unprocessed_jar_path']` with various tools (ready to be dexed). 248 249* `deps_info['host_jar_path']`: 250Path to a file that is the result of processing 251`deps_info['unprocessed_jar_path']` with various tools (use by java_binary). 252 253* `deps_info['interface_jar_path']: 254Path to the interface jar generated for this library. This corresponds to 255a jar file that only contains declarations. Generated by running the `ijar` on 256`deps_info['unprocessed_jar_path']` or the `turbine` tool on source files. 257 258* `deps_info['dex_path']`: 259Path to the `.dex` file generated for this target, from 260`deps_info['device_jar_path']` unless this comes from a prebuilt `.aar` archive. 261 262* `deps_info['is_prebuilt']`: 263True to indicate that this target corresponds to a prebuilt `.jar` file. 264In this case, `deps_info['unprocessed_jar_path']` will point to the source 265`.jar` file. Otherwise, it will be point to a build-generated file. 266 267* `deps_info['target_sources_file']`: 268Path to a single `.sources` file listing all the Java and Kotlin sources that 269were used to generate the library (simple text format, one `.jar` path per 270line). 271 272* `deps_info['lint_android_manifest']`: 273Path to an AndroidManifest.xml file to use for this lint target. 274 275* `deps_info['lint_sources']`: 276The list of all `deps_info['target_sources_file']` entries for all library 277dependencies that are chromium code. Note: this is a list of files, where each 278file contains a list of Java and Kotlin source files. This is used for lint. 279 280* `deps_info['lint_aars']`: 281List of all aars from transitive java dependencies. This allows lint to collect 282their custom annotations.zip and run checks like @IntDef on their annotations. 283 284* `deps_info['lint_srcjars']`: 285List of all bundled srcjars of all transitive java library targets. Excludes 286non-chromium java libraries. 287 288* `deps_info['lint_resource_sources']`: 289List of all resource sources files belonging to all transitive resource 290dependencies of this target. Excludes resources owned by non-chromium code. 291 292* `deps_info['lint_resource_zips']`: 293List of all resource zip files belonging to all transitive resource dependencies 294of this target. Excludes resources owned by non-chromium code. 295 296* `deps_info['javac']`: 297A dictionary containing information about the way the sources in this library 298are compiled. Appears also on other Java-related targets. See the [dedicated 299section about this](#dict_javac) below for details. 300 301* `deps_info['javac_full_classpath']`: 302The classpath used when performing bytecode processing. Essentially the 303collection of all `deps_info['unprocessed_jar_path']` entries for the target 304and all its dependencies. 305 306* `deps_info['javac_full_interface_classpath']`: 307The classpath used when using the errorprone compiler. 308 309* `deps_info['proguard_enabled"]`: 310True to indicate that ProGuard processing is enabled for this target. 311 312* `deps_info['proguard_configs"]`: 313A list of paths to ProGuard configuration files related to this library. 314 315* `deps_info['extra_classpath_jars']: 316For some Java related types, a list of extra `.jar` files to use at build time 317but not at runtime. 318 319## <a name="target_java_binary">Target type `java_binary`</a>: 320 321This type corresponds to a Java binary, which is nothing more than a 322`java_library` target that also provides a main class name. It thus inherits 323all entries from the `java_library` type, and adds: 324 325* `deps_info['main_class']`: 326Name of the main Java class that serves as an entry point for the binary. 327 328* `deps_info['device_classpath']`: 329The classpath used when running a Java or Android binary. Essentially the 330collection of all `deps_info['device_jar_path']` entries for the target and all 331its dependencies. 332 333* `deps_info['all_dex_files']`: 334The list of paths to all `deps_info['dex_path']` entries for all libraries 335that comprise this APK. Valid only for debug builds. 336 337* `deps_info['preferred_dep']`: 338Whether the target should be the preferred dep. This is usually the case when we 339have a java_group that depends on either the public or internal dep accordingly, 340and it is better to depend on the group rather than the underlying dep. Another 341case is for android_library_factory targets, the factory target should be 342preferred instead of the actual implementation. 343 344## <a name="target_robolectric_binary">Target type `robolectric_binary`</a>: 345 346A target type for JUnit-specific binaries. Identical to 347[`java_binary`](#target_java_binary) in the context of `.build_config` files, 348except the name. 349 350 351## <a name="target_java_annotation_processor">Target type \ 352`java_annotation_processor`</a>: 353 354A target type for Java annotation processors. Identical to 355[`java_binary`](#target_java_binary) in the context of `.build_config` files, 356except the name, except that it requires a `deps_info['main_class']` entry. 357 358 359## <a name="target_android_apk">Target type `android_apk`</a>: 360 361Corresponds to an Android APK. Inherits from the 362[`java_binary`](#target_java_binary) type and adds: 363 364* `deps_info['apk_path']`: 365Path to the raw, unsigned, APK generated by this target. 366 367* `deps_info['incremental_apk_path']`: 368Path to the raw, unsigned, incremental APK generated by this target. 369 370* `deps_info['incremental_install_json_path']`: 371Path to the JSON file with per-apk details for incremental install. 372See `build/android/gyp/incremental/write_installer_json.py` for more 373details about its content. 374 375* `deps_info['dist_jar']['all_interface_jars']`: 376For `android_apk` and `dist_jar` targets, a list of all interface jar files 377that will be merged into the final `.jar` file for distribution. 378 379* `deps_info['final_dex']['path']`: 380Path to the final classes.dex file (or classes.zip in case of multi-dex) 381for this APK - only used for proguarded builds. 382 383* `native['libraries']` 384List of native libraries for the primary ABI to be embedded in this APK. 385E.g. [ "libchrome.so" ] (i.e. this doesn't include any ABI sub-directory 386prefix). 387 388* `native['java_libraries_list']` 389The same list as `native['libraries']` as a string holding a Java source 390fragment, e.g. `"{\"chrome\"}"`, without any `lib` prefix, and `.so` 391suffix (as expected by `System.loadLibrary()`). 392 393* `native['second_abi_libraries']` 394List of native libraries for the secondary ABI to be embedded in this APK. 395Empty if only a single ABI is supported. 396 397* `native['loadable_modules']` 398A list of native libraries to store within the APK, in addition to those from 399`native['libraries']`. These correspond to things like the Chromium linker 400or instrumentation libraries. 401 402* `native['secondary_abi_loadable_modules']` 403Secondary ABI version of loadable_modules 404 405* `native['library_always_compress']` 406A list of library files that we always compress. 407 408* `assets` 409A list of assets stored compressed in the APK. Each entry has the format 410`<source-path>:<destination-path>`, where `<source-path>` is relative to 411`$CHROMIUM_OUTPUT_DIR`, and `<destination-path>` is relative to `//assets/` 412within the APK. 413 414NOTE: Not to be confused with the `deps_info['assets']` dictionary that 415belongs to `android_assets` targets only. 416 417* `uncompressed_assets` 418A list of uncompressed assets stored in the APK. Each entry has the format 419`<source-path>:<destination-path>` too. 420 421* `locales_java_list` 422A string holding a Java source fragment that gives the list of locales stored 423uncompressed as android assets. 424 425* `extra_android_manifests` 426A list of `deps_configs['android_manifest]` entries, for all resource 427dependencies for this target. I.e. a list of paths to manifest files for 428all the resources in this APK. These will be merged with the root manifest 429file to generate the final one used to build the APK. 430 431* `java_resources_jars` 432This is a list of `.jar` files whose *Java* resources should be included in 433the final APK. For example, this is used to copy the `.res` files from the 434EMMA Coverage tool. The copy will omit any `.class` file and the top-level 435`//meta-inf/` directory from the input jars. Everything else will be copied 436into the final APK as-is. 437 438NOTE: This has nothing to do with *Android* resources. 439 440* `deps_info['proguard_all_configs']`: 441The collection of all 'deps_info['proguard_configs']` values from this target 442and all its dependencies. 443 444* `deps_info['proguard_classpath_jars']`: 445The collection of all 'deps_info['extra_classpath_jars']` values from all 446dependencies. 447 448* `deps_info['proguard_under_test_mapping']`: 449Applicable to apks with proguard enabled that have an apk_under_test. This is 450the path to the apk_under_test's output proguard .mapping file. 451 452## <a name="target_android_app_bundle_module">Target type \ 453`android_app_bundle_module`</a>: 454 455Corresponds to an Android app bundle module. Very similar to an APK and 456inherits the same fields, except that this does not generate an installable 457file (see `android_app_bundle`), and for the following omitted fields: 458 459* `deps_info['apk_path']`, `deps_info['incremental_apk_path']` and 460 `deps_info['incremental_install_json_path']` are omitted. 461 462* top-level `dist_jar` is omitted as well. 463 464In addition to `android_apk` targets though come these new fields: 465 466* `deps_info['proto_resources_path']`: 467The path of an zip archive containing the APK's resources compiled to the 468protocol buffer format (instead of regular binary xml + resources.arsc). 469 470* `deps_info['module_rtxt_path']`: 471The path of the R.txt file generated when compiling the resources for the bundle 472module. 473 474* `deps_info['module_pathmap_path']`: 475The path of the pathmap file generated when compiling the resources for the 476bundle module, if resource path shortening is enabled. 477 478* `deps_info['base_allowlist_rtxt_path']`: 479Optional path to an R.txt file used as a allowlist for base string resources. 480This means that any string resource listed in this file *and* in 481`deps_info['module_rtxt_path']` will end up in the base split APK of any 482`android_app_bundle` target that uses this target as its base module. 483 484This ensures that such localized strings are available to all bundle installs, 485even when language based splits are enabled (e.g. required for WebView strings 486inside the Monochrome bundle). 487 488 489## <a name="target_android_app_bundle">Target type `android_app_bundle`</a> 490 491This target type corresponds to an Android app bundle, and is built from one 492or more `android_app_bundle_module` targets listed as dependencies. 493 494 495## <a name="target_dist_aar">Target type `dist_aar`</a>: 496 497This type corresponds to a target used to generate an `.aar` archive for 498distribution. The archive's content is determined by the target's dependencies. 499 500This always has the following entries: 501 502 * `deps_info['supports_android']` (always True). 503 * `deps_info['requires_android']` (always True). 504 * `deps_info['proguard_configs']` (optional). 505 506 507## <a name="target_dist_jar">Target type `dist_jar`</a>: 508 509This type is similar to [`dist_aar`](#target_dist_aar) but is not 510Android-specific, and used to create a `.jar` file that can be later 511redistributed. 512 513This always has the following entries: 514 515 * `deps_info['proguard_enabled']` (False by default). 516 * `deps_info['proguard_configs']` (optional). 517 * `deps_info['supports_android']` (True by default). 518 * `deps_info['requires_android']` (False by default). 519 520 521 522## <a name="dict_javac">The `deps_info['javac']` dictionary</a>: 523 524This dictionary appears in Java-related targets (e.g. `java_library`, 525`android_apk` and others), and contains information related to the compilation 526of Java sources, class files, and jars. 527 528* `javac['classpath']` 529The classpath used to compile this target when annotation processors are 530present. 531 532* `javac['interface_classpath']` 533The classpath used to compile this target when annotation processors are 534not present. These are also always used to known when a target needs to be 535rebuilt. 536 537* `javac['processor_classpath']` 538The classpath listing the jars used for annotation processors. I.e. sent as 539`-processorpath` when invoking `javac`. 540 541* `javac['processor_classes']` 542The list of annotation processor main classes. I.e. sent as `-processor' when 543invoking `javac`. 544 545## <a name="android_app_bundle">Target type `android_app_bundle`</a>: 546 547This type corresponds to an Android app bundle (`.aab` file). 548 549--------------- END_MARKDOWN --------------------------------------------------- 550""" 551 552import collections 553import itertools 554import json 555import optparse 556import os 557import shutil 558import sys 559import xml.dom.minidom 560 561from util import build_utils 562from util import resource_utils 563import action_helpers # build_utils adds //build to sys.path. 564 565 566# Types that should never be used as a dependency of another build config. 567_ROOT_TYPES = ('android_apk', 'java_binary', 'java_annotation_processor', 568 'robolectric_binary', 'android_app_bundle') 569# Types that should not allow code deps to pass through. 570_RESOURCE_TYPES = ('android_assets', 'android_resources', 'system_java_library') 571 572# Cache of path -> JSON dict. 573_dep_config_cache = {} 574 575 576class OrderedSet(collections.OrderedDict): 577 @staticmethod 578 def fromkeys(iterable): 579 out = OrderedSet() 580 out.update(iterable) 581 return out 582 583 def add(self, key): 584 self[key] = True 585 586 def remove(self, key): 587 if key in self: 588 del self[key] 589 590 def update(self, iterable): 591 for v in iterable: 592 self.add(v) 593 594 def difference_update(self, iterable): 595 for v in iterable: 596 self.remove(v) 597 598 599def _ExtractMarkdownDocumentation(input_text): 600 """Extract Markdown documentation from a list of input strings lines. 601 602 This generates a list of strings extracted from |input_text|, by looking 603 for '-- BEGIN_MARKDOWN --' and '-- END_MARKDOWN --' line markers.""" 604 in_markdown = False 605 result = [] 606 for line in input_text.splitlines(): 607 if in_markdown: 608 if '-- END_MARKDOWN --' in line: 609 in_markdown = False 610 else: 611 result.append(line) 612 else: 613 if '-- BEGIN_MARKDOWN --' in line: 614 in_markdown = True 615 616 return result 617 618 619class AndroidManifest: 620 def __init__(self, path): 621 self.path = path 622 dom = xml.dom.minidom.parse(path) 623 manifests = dom.getElementsByTagName('manifest') 624 assert len(manifests) == 1 625 self.manifest = manifests[0] 626 627 def GetInstrumentationElements(self): 628 instrumentation_els = self.manifest.getElementsByTagName('instrumentation') 629 if len(instrumentation_els) == 0: 630 return None 631 return instrumentation_els 632 633 def CheckInstrumentationElements(self, expected_package): 634 instrs = self.GetInstrumentationElements() 635 if not instrs: 636 raise Exception('No <instrumentation> elements found in %s' % self.path) 637 for instr in instrs: 638 instrumented_package = instr.getAttributeNS( 639 'http://schemas.android.com/apk/res/android', 'targetPackage') 640 if instrumented_package != expected_package: 641 raise Exception( 642 'Wrong instrumented package. Expected %s, got %s' 643 % (expected_package, instrumented_package)) 644 645 def GetPackageName(self): 646 return self.manifest.getAttribute('package') 647 648 649def GetDepConfigRoot(path): 650 if not path in _dep_config_cache: 651 with open(path) as jsonfile: 652 _dep_config_cache[path] = json.load(jsonfile) 653 return _dep_config_cache[path] 654 655 656def GetDepConfig(path): 657 return GetDepConfigRoot(path)['deps_info'] 658 659 660def DepsOfType(wanted_type, configs): 661 return [c for c in configs if c['type'] == wanted_type] 662 663 664def DepPathsOfType(wanted_type, config_paths): 665 return [p for p in config_paths if GetDepConfig(p)['type'] == wanted_type] 666 667 668def GetAllDepsConfigsInOrder(deps_config_paths, filter_func=None): 669 def apply_filter(paths): 670 if filter_func: 671 return [p for p in paths if filter_func(GetDepConfig(p))] 672 return paths 673 674 def discover(path): 675 config = GetDepConfig(path) 676 all_deps = config['deps_configs'] + config.get('public_deps_configs', []) 677 return apply_filter(all_deps) 678 679 deps_config_paths = apply_filter(deps_config_paths) 680 deps_config_paths = build_utils.GetSortedTransitiveDependencies( 681 deps_config_paths, discover) 682 return deps_config_paths 683 684 685def GetObjectByPath(obj, key_path): 686 """Given an object, return its nth child based on a key path. 687 """ 688 return GetObjectByPath(obj[key_path[0]], key_path[1:]) if key_path else obj 689 690 691def RemoveObjDups(obj, base, *key_path): 692 """Remove array items from an object[*kep_path] that are also 693 contained in the base[*kep_path] (duplicates). 694 """ 695 base_target = set(GetObjectByPath(base, key_path)) 696 target = GetObjectByPath(obj, key_path) 697 target[:] = [x for x in target if x not in base_target] 698 699 700class Deps: 701 def __init__(self, direct_deps_config_paths): 702 self._all_deps_config_paths = GetAllDepsConfigsInOrder( 703 direct_deps_config_paths) 704 self._direct_deps_configs = [ 705 GetDepConfig(p) for p in direct_deps_config_paths 706 ] 707 self._all_deps_configs = [ 708 GetDepConfig(p) for p in self._all_deps_config_paths 709 ] 710 self._direct_deps_config_paths = direct_deps_config_paths 711 712 def All(self, wanted_type=None): 713 if wanted_type is None: 714 return self._all_deps_configs 715 return DepsOfType(wanted_type, self._all_deps_configs) 716 717 def Direct(self, wanted_type=None): 718 if wanted_type is None: 719 return self._direct_deps_configs 720 return DepsOfType(wanted_type, self._direct_deps_configs) 721 722 def AllConfigPaths(self): 723 return self._all_deps_config_paths 724 725 def GradlePrebuiltJarPaths(self): 726 ret = [] 727 728 def helper(cur): 729 for config in cur.Direct('java_library'): 730 if config['is_prebuilt'] or config['gradle_treat_as_prebuilt']: 731 if config['unprocessed_jar_path'] not in ret: 732 ret.append(config['unprocessed_jar_path']) 733 734 helper(self) 735 return ret 736 737 def GradleLibraryProjectDeps(self): 738 ret = [] 739 740 def helper(cur): 741 for config in cur.Direct('java_library'): 742 if config['is_prebuilt']: 743 pass 744 elif config['gradle_treat_as_prebuilt']: 745 all_deps = config['deps_configs'] + config.get( 746 'public_deps_configs', []) 747 helper(Deps(all_deps)) 748 elif config not in ret: 749 ret.append(config) 750 751 helper(self) 752 return ret 753 754 755def _MergeAssets(all_assets): 756 """Merges all assets from the given deps. 757 758 Returns: 759 A tuple of: (compressed, uncompressed, locale_paks) 760 |compressed| and |uncompressed| are lists of "srcPath:zipPath". srcPath is 761 the path of the asset to add, and zipPath is the location within the zip 762 (excluding assets/ prefix). 763 |locale_paks| is a set of all zipPaths that have been marked as 764 treat_as_locale_paks=true. 765 """ 766 compressed = {} 767 uncompressed = {} 768 locale_paks = set() 769 for asset_dep in all_assets: 770 entry = asset_dep['assets'] 771 disable_compression = entry.get('disable_compression') 772 treat_as_locale_paks = entry.get('treat_as_locale_paks') 773 dest_map = uncompressed if disable_compression else compressed 774 other_map = compressed if disable_compression else uncompressed 775 outputs = entry.get('outputs', []) 776 for src, dest in itertools.zip_longest(entry['sources'], outputs): 777 if not dest: 778 dest = os.path.basename(src) 779 # Merge so that each path shows up in only one of the lists, and that 780 # deps of the same target override previous ones. 781 other_map.pop(dest, 0) 782 dest_map[dest] = src 783 if treat_as_locale_paks: 784 locale_paks.add(dest) 785 786 def create_list(asset_map): 787 # Sort to ensure deterministic ordering. 788 items = sorted(asset_map.items()) 789 return [f'{src}:{dest}' for dest, src in items] 790 791 return create_list(compressed), create_list(uncompressed), locale_paks 792 793 794def _SuffixAssets(suffix_names, suffix, assets): 795 new_assets = [] 796 for x in assets: 797 src_path, apk_subpath = x.split(':', 1) 798 if apk_subpath in suffix_names: 799 apk_subpath += suffix 800 new_assets.append(f'{src_path}:{apk_subpath}') 801 return new_assets 802 803 804def _ResolveGroupsAndPublicDeps(config_paths): 805 """Returns a list of configs with all groups inlined.""" 806 807 def helper(config_path): 808 config = GetDepConfig(config_path) 809 if config['type'] == 'group': 810 # Groups combine public_deps with deps_configs, so no need to check 811 # public_config_paths separately. 812 return config['deps_configs'] 813 if config['type'] == 'android_resources': 814 # android_resources targets do not support public_deps, but instead treat 815 # all resource deps as public deps. 816 return DepPathsOfType('android_resources', config['deps_configs']) 817 818 return config.get('public_deps_configs', []) 819 820 return build_utils.GetSortedTransitiveDependencies(config_paths, helper) 821 822 823def _DepsFromPaths(dep_paths, 824 target_type, 825 filter_root_targets=True, 826 recursive_resource_deps=False): 827 """Resolves all groups and trims dependency branches that we never want. 828 829 E.g. When a resource or asset depends on an apk target, the intent is to 830 include the .apk as a resource/asset, not to have the apk's classpath added. 831 832 This method is meant to be called to get the top nodes (i.e. closest to 833 current target) that we could then use to get a full transitive dependants 834 list (eg using Deps#all). So filtering single elements out of this list, 835 filters whole branches of dependencies. By resolving groups (i.e. expanding 836 them to their constituents), depending on a group is equivalent to directly 837 depending on each element of that group. 838 """ 839 blocklist = [] 840 allowlist = [] 841 842 # Don't allow root targets to be considered as a dep. 843 if filter_root_targets: 844 blocklist.extend(_ROOT_TYPES) 845 846 # Don't allow java libraries to cross through assets/resources. 847 if target_type in _RESOURCE_TYPES: 848 allowlist.extend(_RESOURCE_TYPES) 849 # Pretend that this target directly depends on all of its transitive 850 # dependencies. 851 if recursive_resource_deps: 852 dep_paths = GetAllDepsConfigsInOrder(dep_paths) 853 # Exclude assets if recursive_resource_deps is set. The 854 # recursive_resource_deps arg is used to pull resources into the base 855 # module to workaround bugs accessing resources in isolated DFMs, but 856 # assets should be kept in the DFMs. 857 blocklist.append('android_assets') 858 859 return _DepsFromPathsWithFilters(dep_paths, blocklist, allowlist) 860 861 862def _FilterConfigPaths(dep_paths, blocklist=None, allowlist=None): 863 if not blocklist and not allowlist: 864 return dep_paths 865 configs = [GetDepConfig(p) for p in dep_paths] 866 if blocklist: 867 configs = [c for c in configs if c['type'] not in blocklist] 868 if allowlist: 869 configs = [c for c in configs if c['type'] in allowlist] 870 871 return [c['path'] for c in configs] 872 873 874def _DepsFromPathsWithFilters(dep_paths, blocklist=None, allowlist=None): 875 """Resolves all groups and trims dependency branches that we never want. 876 877 See _DepsFromPaths. 878 879 |blocklist| if passed, are the types of direct dependencies we do not care 880 about (i.e. tips of branches that we wish to prune). 881 882 |allowlist| if passed, are the only types of direct dependencies we care 883 about (i.e. we wish to prune all other branches that do not start from one of 884 these). 885 """ 886 # Filter both before and after so that public_deps of blocked targets are not 887 # added. 888 allowlist_with_groups = None 889 if allowlist: 890 allowlist_with_groups = set(allowlist) 891 allowlist_with_groups.add('group') 892 dep_paths = _FilterConfigPaths(dep_paths, blocklist, allowlist_with_groups) 893 dep_paths = _ResolveGroupsAndPublicDeps(dep_paths) 894 dep_paths = _FilterConfigPaths(dep_paths, blocklist, allowlist) 895 896 return Deps(dep_paths) 897 898 899def _ExtractSharedLibsFromRuntimeDeps(runtime_deps_file): 900 ret = [] 901 with open(runtime_deps_file) as f: 902 for line in f: 903 line = line.rstrip() 904 if not line.endswith('.so'): 905 continue 906 # Only unstripped .so files are listed in runtime deps. 907 # Convert to the stripped .so by going up one directory. 908 ret.append(os.path.normpath(line.replace('lib.unstripped/', ''))) 909 ret.reverse() 910 return ret 911 912 913def _CreateJavaLibrariesList(library_paths): 914 """Returns a java literal array with the "base" library names: 915 e.g. libfoo.so -> foo 916 """ 917 names = ['"%s"' % os.path.basename(s)[3:-3] for s in library_paths] 918 return ('{%s}' % ','.join(sorted(set(names)))) 919 920 921def _CreateJavaLocaleListFromAssets(assets, locale_paks): 922 """Returns a java literal array from a list of locale assets. 923 924 Args: 925 assets: A list of all APK asset paths in the form 'src:dst' 926 locale_paks: A list of asset paths that correponds to the locale pak 927 files of interest. Each |assets| entry will have its 'dst' part matched 928 against it to determine if they are part of the result. 929 Returns: 930 A string that is a Java source literal array listing the locale names 931 of the corresponding asset files, without directory or .pak suffix. 932 E.g. '{"en-GB", "en-US", "es-ES", "fr", ... }' 933 """ 934 assets_paths = [a.split(':')[1] for a in assets] 935 locales = [os.path.basename(a)[:-4] for a in assets_paths if a in locale_paks] 936 return '{%s}' % ','.join('"%s"' % l for l in sorted(locales)) 937 938 939def _AddJarMapping(jar_to_target, configs): 940 for config in configs: 941 jar = config.get('unprocessed_jar_path') 942 if jar: 943 jar_to_target[jar] = config['gn_target'] 944 for jar in config.get('extra_classpath_jars', []): 945 jar_to_target[jar] = config['gn_target'] 946 947 948def _CompareClasspathPriority(dep): 949 return 1 if dep.get('low_classpath_priority') else 0 950 951 952def _DedupFeatureModuleSharedCode(uses_split_arg, modules, 953 field_names_to_dedup): 954 child_to_ancestors = collections.defaultdict(list) 955 if uses_split_arg: 956 for split_pair in uses_split_arg: 957 child, parent = split_pair.split(':') 958 assert child in modules 959 assert parent in modules 960 child_to_ancestors[child] = [parent] 961 962 # Create a full list of ancestors for each module. 963 for name in modules: 964 if name == 'base': 965 continue 966 curr_name = name 967 while curr_name in child_to_ancestors: 968 parent = child_to_ancestors[curr_name][0] 969 if parent not in child_to_ancestors[name]: 970 child_to_ancestors[name].append(parent) 971 curr_name = parent 972 973 if curr_name != 'base': 974 child_to_ancestors[name].append('base') 975 976 # Strip out duplicates from ancestors. 977 for name, module in modules.items(): 978 if name == 'base': 979 continue 980 # Make sure we get all ancestors, not just direct parent. 981 for ancestor in child_to_ancestors[name]: 982 for f in field_names_to_dedup: 983 if f in module: 984 RemoveObjDups(module, modules[ancestor], f) 985 986 # Strip out duplicates from siblings/cousins. 987 for f in field_names_to_dedup: 988 _PromoteToCommonAncestor(modules, child_to_ancestors, f) 989 990 991def _PromoteToCommonAncestor(modules, child_to_ancestors, field_name): 992 module_to_fields_set = {} 993 for module_name, module in modules.items(): 994 if field_name in module: 995 module_to_fields_set[module_name] = set(module[field_name]) 996 997 seen = set() 998 dupes = set() 999 for fields in module_to_fields_set.values(): 1000 new_dupes = seen & fields 1001 if new_dupes: 1002 dupes |= new_dupes 1003 seen |= fields 1004 1005 for d in dupes: 1006 owning_modules = [] 1007 for module_name, fields in module_to_fields_set.items(): 1008 if d in fields: 1009 owning_modules.append(module_name) 1010 assert len(owning_modules) >= 2 1011 # Rely on the fact that ancestors are inserted from closest to 1012 # farthest, where "base" should always be the last element. 1013 # Arbitrarily using the first owning module - any would work. 1014 for ancestor in child_to_ancestors[owning_modules[0]]: 1015 ancestor_is_shared_with_all = True 1016 for o in owning_modules[1:]: 1017 if ancestor not in child_to_ancestors[o]: 1018 ancestor_is_shared_with_all = False 1019 break 1020 if ancestor_is_shared_with_all: 1021 common_ancestor = ancestor 1022 break 1023 for o in owning_modules: 1024 module_to_fields_set[o].remove(d) 1025 module_to_fields_set[common_ancestor].add(d) 1026 1027 for module_name, module in modules.items(): 1028 if field_name in module: 1029 module[field_name] = sorted(list(module_to_fields_set[module_name])) 1030 1031 1032def _CopyBuildConfigsForDebugging(debug_dir): 1033 shutil.rmtree(debug_dir, ignore_errors=True) 1034 os.makedirs(debug_dir) 1035 for src_path in _dep_config_cache: 1036 dst_path = os.path.join(debug_dir, src_path) 1037 assert dst_path.startswith(debug_dir), dst_path 1038 os.makedirs(os.path.dirname(dst_path), exist_ok=True) 1039 shutil.copy(src_path, dst_path) 1040 print(f'Copied {len(_dep_config_cache)} .build_config.json into {debug_dir}') 1041 1042 1043def main(argv): 1044 parser = optparse.OptionParser() 1045 action_helpers.add_depfile_arg(parser) 1046 parser.add_option('--build-config', help='Path to build_config output.') 1047 parser.add_option('--store-deps-for-debugging-to', 1048 help='Path to copy all transitive build config files to.') 1049 parser.add_option( 1050 '--type', 1051 help='Type of this target (e.g. android_library).') 1052 parser.add_option('--gn-target', help='GN label for this target') 1053 parser.add_option( 1054 '--deps-configs', 1055 help='GN-list of dependent build_config files.') 1056 parser.add_option( 1057 '--annotation-processor-configs', 1058 help='GN-list of build_config files for annotation processors.') 1059 1060 # android_resources options 1061 parser.add_option('--srcjar', help='Path to target\'s resources srcjar.') 1062 parser.add_option('--resources-zip', help='Path to target\'s resources zip.') 1063 parser.add_option('--package-name', 1064 help='Java package name for these resources.') 1065 parser.add_option('--android-manifest', 1066 help='Path to the root android manifest.') 1067 parser.add_option('--merged-android-manifest', 1068 help='Path to the merged android manifest.') 1069 parser.add_option('--resource-dirs', action='append', default=[], 1070 help='GYP-list of resource dirs') 1071 parser.add_option( 1072 '--res-sources-path', 1073 help='Path to file containing a list of paths to resources.') 1074 parser.add_option( 1075 '--resource-overlay', 1076 action='store_true', 1077 help='Whether resources passed in via --resources-zip should override ' 1078 'resources with the same name') 1079 parser.add_option( 1080 '--recursive-resource-deps', 1081 action='store_true', 1082 help='Whether deps should be walked recursively to find resource deps.') 1083 1084 # android_assets options 1085 parser.add_option('--asset-sources', help='List of asset sources.') 1086 parser.add_option('--asset-renaming-sources', 1087 help='List of asset sources with custom destinations.') 1088 parser.add_option('--asset-renaming-destinations', 1089 help='List of asset custom destinations.') 1090 parser.add_option('--disable-asset-compression', action='store_true', 1091 help='Whether to disable asset compression.') 1092 parser.add_option('--treat-as-locale-paks', action='store_true', 1093 help='Consider the assets as locale paks in BuildConfig.java') 1094 1095 # java library and group options 1096 parser.add_option('--preferred-dep', 1097 action='store_true', 1098 help='Whether the target should be preferred as a dep.') 1099 1100 # java library options 1101 parser.add_option('--public-deps-configs', 1102 help='GN list of config files of deps which are exposed as ' 1103 'part of the target\'s public API.') 1104 parser.add_option('--aar-path', help='Path to containing .aar file.') 1105 parser.add_option('--device-jar-path', help='Path to .jar for dexing.') 1106 parser.add_option('--host-jar-path', help='Path to .jar for java_binary.') 1107 parser.add_option('--unprocessed-jar-path', 1108 help='Path to the .jar to use for javac classpath purposes.') 1109 parser.add_option( 1110 '--interface-jar-path', 1111 help='Path to the interface .jar to use for javac classpath purposes.') 1112 parser.add_option('--is-prebuilt', action='store_true', 1113 help='Whether the jar was compiled or pre-compiled.') 1114 parser.add_option('--target-sources-file', help='Path to .sources file') 1115 parser.add_option('--bundled-srcjars', 1116 help='GYP-list of .srcjars that have been included in this java_library.') 1117 parser.add_option('--supports-android', action='store_true', 1118 help='Whether this library supports running on the Android platform.') 1119 parser.add_option('--requires-android', action='store_true', 1120 help='Whether this library requires running on the Android platform.') 1121 parser.add_option('--bypass-platform-checks', action='store_true', 1122 help='Bypass checks for support/require Android platform.') 1123 parser.add_option('--extra-classpath-jars', 1124 help='GYP-list of .jar files to include on the classpath when compiling, ' 1125 'but not to include in the final binary.') 1126 parser.add_option( 1127 '--low-classpath-priority', 1128 action='store_true', 1129 help='Indicates that the library should be placed at the end of the ' 1130 'classpath.') 1131 parser.add_option( 1132 '--mergeable-android-manifests', 1133 help='GN-list of AndroidManifest.xml to include in manifest merging.') 1134 parser.add_option('--gradle-treat-as-prebuilt', action='store_true', 1135 help='Whether this library should be treated as a prebuilt library by ' 1136 'generate_gradle.py.') 1137 parser.add_option('--main-class', 1138 help='Main class for java_binary or java_annotation_processor targets.') 1139 parser.add_option('--java-resources-jar-path', 1140 help='Path to JAR that contains java resources. Everything ' 1141 'from this JAR except meta-inf/ content and .class files ' 1142 'will be added to the final APK.') 1143 parser.add_option( 1144 '--non-chromium-code', 1145 action='store_true', 1146 help='True if a java library is not chromium code, used for lint.') 1147 1148 # robolectric_library options 1149 parser.add_option('--is-robolectric', 1150 action='store_true', 1151 help='Whether this is a host side android test library.') 1152 1153 # android library options 1154 parser.add_option('--dex-path', help='Path to target\'s dex output.') 1155 1156 # native library options 1157 parser.add_option('--shared-libraries-runtime-deps', 1158 help='Path to file containing runtime deps for shared ' 1159 'libraries.') 1160 parser.add_option( 1161 '--loadable-modules', 1162 action='append', 1163 help='GN-list of native libraries for primary ' 1164 'android-abi. Can be specified multiple times.', 1165 default=[]) 1166 parser.add_option('--secondary-abi-shared-libraries-runtime-deps', 1167 help='Path to file containing runtime deps for secondary ' 1168 'abi shared libraries.') 1169 parser.add_option( 1170 '--secondary-abi-loadable-modules', 1171 action='append', 1172 help='GN-list of native libraries for secondary ' 1173 'android-abi. Can be specified multiple times.', 1174 default=[]) 1175 parser.add_option( 1176 '--native-lib-placeholders', 1177 action='append', 1178 help='GN-list of native library placeholders to add.', 1179 default=[]) 1180 parser.add_option( 1181 '--secondary-native-lib-placeholders', 1182 action='append', 1183 help='GN-list of native library placeholders to add ' 1184 'for the secondary android-abi.', 1185 default=[]) 1186 parser.add_option('--uncompress-shared-libraries', default=False, 1187 action='store_true', 1188 help='Whether to store native libraries uncompressed') 1189 parser.add_option( 1190 '--library-always-compress', 1191 help='The list of library files that we always compress.') 1192 1193 # apk options 1194 parser.add_option('--apk-path', help='Path to the target\'s apk output.') 1195 parser.add_option('--incremental-apk-path', 1196 help="Path to the target's incremental apk output.") 1197 parser.add_option('--incremental-install-json-path', 1198 help="Path to the target's generated incremental install " 1199 "json.") 1200 parser.add_option( 1201 '--suffix-apk-assets-used-by', 1202 help='Path to the build config of the apk whose asset list should be ' 1203 'suffixed.') 1204 parser.add_option( 1205 '--tested-apk-config', 1206 help='Path to the build config of the tested apk (for an instrumentation ' 1207 'test apk).') 1208 parser.add_option( 1209 '--proguard-enabled', 1210 action='store_true', 1211 help='Whether proguard is enabled for this apk or bundle module.') 1212 parser.add_option( 1213 '--proguard-configs', 1214 help='GN-list of proguard flag files to use in final apk.') 1215 parser.add_option( 1216 '--proguard-mapping-path', help='Path to jar created by ProGuard step') 1217 1218 # apk options that are static library specific 1219 parser.add_option( 1220 '--static-library-dependent-configs', 1221 help='GN list of .build_configs of targets that use this target as a ' 1222 'static library.') 1223 1224 # options shared between android_resources and apk targets 1225 parser.add_option('--r-text-path', help='Path to target\'s R.txt file.') 1226 1227 parser.add_option('--fail', 1228 help='GN-list of error message lines to fail with.') 1229 1230 parser.add_option('--final-dex-path', 1231 help='Path to final input classes.dex (or classes.zip) to ' 1232 'use in final apk.') 1233 parser.add_option('--res-size-info', help='Path to .ap_.info') 1234 parser.add_option('--apk-proto-resources', 1235 help='Path to resources compiled in protocol buffer format ' 1236 ' for this apk.') 1237 parser.add_option( 1238 '--module-pathmap-path', 1239 help='Path to pathmap file for resource paths in a bundle module.') 1240 parser.add_option( 1241 '--base-allowlist-rtxt-path', 1242 help='Path to R.txt file for the base resources allowlist.') 1243 1244 parser.add_option('--generate-markdown-format-doc', action='store_true', 1245 help='Dump the Markdown .build_config format documentation ' 1246 'then exit immediately.') 1247 1248 parser.add_option('--module-name', help='The name of this feature module.') 1249 parser.add_option( 1250 '--base-module-build-config', 1251 help='Path to the base module\'s build config ' 1252 'if this is a feature module.') 1253 parser.add_option('--parent-module-build-config', 1254 help='Path to the parent module\'s build config ' 1255 'when not using base module as parent.') 1256 1257 parser.add_option( 1258 '--module-build-configs', 1259 help='For bundles, the paths of all non-async module .build_configs ' 1260 'for modules that are part of the bundle.') 1261 parser.add_option( 1262 '--uses-split', 1263 action='append', 1264 help='List of name pairs separated by : mapping a feature module to a ' 1265 'dependent feature module.') 1266 1267 parser.add_option( 1268 '--trace-events-jar-dir', 1269 help='Directory of rewritten .jar files for trace event rewriting.') 1270 1271 parser.add_option('--version-name', help='Version name for this APK.') 1272 parser.add_option('--version-code', help='Version code for this APK.') 1273 1274 options, args = parser.parse_args(argv) 1275 1276 if args: 1277 parser.error('No positional arguments should be given.') 1278 1279 if options.generate_markdown_format_doc: 1280 doc_lines = _ExtractMarkdownDocumentation(__doc__) 1281 for line in doc_lines: 1282 print(line) 1283 return 0 1284 1285 if options.fail: 1286 parser.error('\n'.join(action_helpers.parse_gn_list(options.fail))) 1287 1288 lib_options = ['unprocessed_jar_path', 'interface_jar_path'] 1289 device_lib_options = ['device_jar_path', 'dex_path'] 1290 required_options_map = { 1291 'android_apk': ['build_config'] + lib_options + device_lib_options, 1292 'android_app_bundle_module': 1293 ['build_config', 'res_size_info'] + lib_options + device_lib_options, 1294 'android_assets': ['build_config'], 1295 'android_resources': ['build_config', 'resources_zip'], 1296 'dist_aar': ['build_config'], 1297 'dist_jar': ['build_config'], 1298 'group': ['build_config'], 1299 'java_annotation_processor': ['build_config', 'main_class'], 1300 'java_binary': ['build_config'], 1301 'java_library': ['build_config', 'host_jar_path'] + lib_options, 1302 'robolectric_binary': ['build_config'], 1303 'system_java_library': ['build_config', 'unprocessed_jar_path'], 1304 'android_app_bundle': ['build_config', 'module_build_configs'], 1305 } 1306 required_options = required_options_map.get(options.type) 1307 if not required_options: 1308 raise Exception('Unknown type: <%s>' % options.type) 1309 1310 build_utils.CheckOptions(options, parser, required_options) 1311 1312 if options.type != 'android_app_bundle_module': 1313 if options.apk_proto_resources: 1314 raise Exception('--apk-proto-resources can only be used with ' 1315 '--type=android_app_bundle_module') 1316 if options.module_pathmap_path: 1317 raise Exception('--module-pathmap-path can only be used with ' 1318 '--type=android_app_bundle_module') 1319 if options.base_allowlist_rtxt_path: 1320 raise Exception('--base-allowlist-rtxt-path can only be used with ' 1321 '--type=android_app_bundle_module') 1322 if options.module_name: 1323 raise Exception('--module-name can only be used with ' 1324 '--type=android_app_bundle_module') 1325 1326 is_apk_or_module_target = options.type in ('android_apk', 1327 'android_app_bundle_module') 1328 1329 if not is_apk_or_module_target: 1330 if options.library_always_compress: 1331 raise Exception( 1332 '--library-always-compress can only be used with --type=android_apk ' 1333 'or --type=android_app_bundle_module') 1334 1335 if options.device_jar_path and not options.dex_path: 1336 raise Exception('java_library that supports Android requires a dex path.') 1337 if any(getattr(options, x) for x in lib_options): 1338 for attr in lib_options: 1339 if not getattr(options, attr): 1340 raise('Expected %s to be set.' % attr) 1341 1342 if options.requires_android and not options.supports_android: 1343 raise Exception( 1344 '--supports-android is required when using --requires-android') 1345 1346 is_java_target = options.type in ('java_binary', 'robolectric_binary', 1347 'java_annotation_processor', 'java_library', 1348 'android_apk', 'dist_aar', 'dist_jar', 1349 'system_java_library', 1350 'android_app_bundle_module') 1351 1352 deps_configs_paths = action_helpers.parse_gn_list(options.deps_configs) 1353 public_deps_configs_paths = action_helpers.parse_gn_list( 1354 options.public_deps_configs) 1355 deps_configs_paths += public_deps_configs_paths 1356 deps = _DepsFromPaths(deps_configs_paths, 1357 options.type, 1358 recursive_resource_deps=options.recursive_resource_deps) 1359 public_deps = _DepsFromPaths(public_deps_configs_paths, options.type) 1360 processor_deps = _DepsFromPaths(action_helpers.parse_gn_list( 1361 options.annotation_processor_configs or ''), 1362 options.type, 1363 filter_root_targets=False) 1364 1365 all_inputs = (deps.AllConfigPaths() + processor_deps.AllConfigPaths()) 1366 1367 if options.recursive_resource_deps: 1368 # Include java_library targets since changes to these targets can remove 1369 # resource deps from the build, which would require rebuilding this target's 1370 # build config file: crbug.com/1168655. 1371 recursive_java_deps = _DepsFromPathsWithFilters( 1372 GetAllDepsConfigsInOrder(deps_configs_paths), 1373 allowlist=['java_library']) 1374 all_inputs.extend(recursive_java_deps.AllConfigPaths()) 1375 1376 system_library_deps = deps.Direct('system_java_library') 1377 all_deps = deps.All() 1378 all_library_deps = deps.All('java_library') 1379 1380 direct_resources_deps = deps.Direct('android_resources') 1381 if options.type == 'java_library': 1382 # For Java libraries, restrict to resource targets that are direct deps, or 1383 # are indirect via other resource targets. 1384 # The indirect-through-other-targets ones are picked up because 1385 # _ResolveGroupsAndPublicDeps() treats resource deps of resource targets as 1386 # public_deps. 1387 all_resources_deps = direct_resources_deps 1388 else: 1389 all_resources_deps = deps.All('android_resources') 1390 1391 if options.type == 'android_resources' and options.recursive_resource_deps: 1392 # android_resources targets that want recursive resource deps also need to 1393 # collect package_names from all library deps. This ensures the R.java files 1394 # for these libraries will get pulled in along with the resources. 1395 android_resources_library_deps = _DepsFromPathsWithFilters( 1396 deps_configs_paths, allowlist=['java_library']).All('java_library') 1397 1398 base_module_build_config = None 1399 if options.base_module_build_config: 1400 base_module_build_config = GetDepConfigRoot( 1401 options.base_module_build_config) 1402 parent_module_build_config = base_module_build_config 1403 if options.parent_module_build_config: 1404 parent_module_build_config = GetDepConfigRoot( 1405 options.parent_module_build_config) 1406 1407 # Initialize some common config. 1408 # Any value that needs to be queryable by dependents must go within deps_info. 1409 config = { 1410 'deps_info': { 1411 'name': os.path.basename(options.build_config), 1412 'path': options.build_config, 1413 'type': options.type, 1414 'gn_target': options.gn_target, 1415 'chromium_code': not options.non_chromium_code, 1416 }, 1417 # Info needed only by generate_gradle.py. 1418 'gradle': {} 1419 } 1420 deps_info = config['deps_info'] 1421 gradle = config['gradle'] 1422 1423 # The paths we record as deps can differ from deps_config_paths: 1424 # 1) Paths can be removed when blocked by _ROOT_TYPES / _RESOURCE_TYPES. 1425 # 2) Paths can be added when promoted from group deps or public_deps of deps. 1426 # Deps are promoted from groups/public_deps in order to make the filtering 1427 # of 1) work through group() targets (which themselves are not resource 1428 # targets, but should be treated as such when depended on by a resource 1429 # target. A more involved filtering implementation could work to maintain 1430 # the semantics of 1) without the need to promote deps, but we've avoided 1431 # such an undertaking so far. 1432 public_deps_set = set() 1433 if public_deps_configs_paths: 1434 deps_info['public_deps_configs'] = [d['path'] for d in public_deps.Direct()] 1435 public_deps_set = set(deps_info['public_deps_configs']) 1436 1437 deps_info['deps_configs'] = [ 1438 d['path'] for d in deps.Direct() if d['path'] not in public_deps_set 1439 ] 1440 1441 if options.type == 'android_apk' and options.tested_apk_config: 1442 tested_apk_deps = Deps([options.tested_apk_config]) 1443 tested_apk_config = tested_apk_deps.Direct()[0] 1444 gradle['apk_under_test'] = tested_apk_config['name'] 1445 1446 if options.type == 'android_app_bundle_module': 1447 deps_info['module_name'] = options.module_name 1448 1449 # Required for generating gradle files. 1450 if options.type == 'java_library': 1451 deps_info['is_prebuilt'] = bool(options.is_prebuilt) 1452 deps_info['gradle_treat_as_prebuilt'] = options.gradle_treat_as_prebuilt 1453 1454 if options.preferred_dep: 1455 deps_info['preferred_dep'] = bool(options.preferred_dep) 1456 1457 if options.android_manifest: 1458 deps_info['android_manifest'] = options.android_manifest 1459 1460 if options.merged_android_manifest: 1461 deps_info['merged_android_manifest'] = options.merged_android_manifest 1462 1463 if options.bundled_srcjars: 1464 deps_info['bundled_srcjars'] = action_helpers.parse_gn_list( 1465 options.bundled_srcjars) 1466 1467 if options.target_sources_file: 1468 deps_info['target_sources_file'] = options.target_sources_file 1469 1470 if is_java_target: 1471 if options.main_class: 1472 deps_info['main_class'] = options.main_class 1473 1474 dependent_prebuilt_jars = deps.GradlePrebuiltJarPaths() 1475 dependent_prebuilt_jars.sort() 1476 if dependent_prebuilt_jars: 1477 gradle['dependent_prebuilt_jars'] = dependent_prebuilt_jars 1478 1479 dependent_android_projects = [] 1480 dependent_java_projects = [] 1481 for c in deps.GradleLibraryProjectDeps(): 1482 if c['requires_android']: 1483 dependent_android_projects.append(c['path']) 1484 else: 1485 dependent_java_projects.append(c['path']) 1486 1487 gradle['dependent_android_projects'] = dependent_android_projects 1488 gradle['dependent_java_projects'] = dependent_java_projects 1489 1490 if options.r_text_path: 1491 deps_info['r_text_path'] = options.r_text_path 1492 1493 if is_apk_or_module_target or options.type in ('group', 'java_library', 1494 'robolectric_binary'): 1495 if options.apk_proto_resources: 1496 deps_info['proto_resources_path'] = options.apk_proto_resources 1497 1498 deps_info['version_name'] = options.version_name 1499 deps_info['version_code'] = options.version_code 1500 if options.module_pathmap_path: 1501 deps_info['module_pathmap_path'] = options.module_pathmap_path 1502 else: 1503 # Ensure there is an entry, even if it is empty, for modules 1504 # that have not enabled resource path shortening. Otherwise 1505 # build_utils.ExpandFileArgs fails. 1506 deps_info['module_pathmap_path'] = '' 1507 1508 if options.base_allowlist_rtxt_path: 1509 deps_info['base_allowlist_rtxt_path'] = options.base_allowlist_rtxt_path 1510 else: 1511 # Ensure there is an entry, even if it is empty, for modules 1512 # that don't need such a allowlist. 1513 deps_info['base_allowlist_rtxt_path'] = '' 1514 1515 if is_java_target: 1516 deps_info['requires_android'] = bool(options.requires_android) 1517 deps_info['supports_android'] = bool(options.supports_android) 1518 1519 # robolectric is special in that its an android target that runs on host. 1520 # You are allowed to depend on both android |deps_require_android| and 1521 # non-android |deps_not_support_android| targets. 1522 if not options.bypass_platform_checks and not options.is_robolectric: 1523 deps_require_android = direct_resources_deps + [ 1524 d for d in deps.Direct() if d.get('requires_android', False) 1525 ] 1526 deps_not_support_android = [ 1527 d for d in deps.Direct() if not d.get('supports_android', True) 1528 ] 1529 1530 if deps_require_android and not options.requires_android: 1531 raise Exception( 1532 'Some deps require building for the Android platform:\n' + 1533 '\n'.join('* ' + d['gn_target'] for d in deps_require_android)) 1534 1535 if deps_not_support_android and options.supports_android: 1536 raise Exception('Not all deps support the Android platform:\n' + 1537 '\n'.join('* ' + d['gn_target'] 1538 for d in deps_not_support_android)) 1539 1540 if is_apk_or_module_target or options.type == 'dist_jar': 1541 all_dex_files = [c['dex_path'] for c in all_library_deps] 1542 1543 if is_java_target: 1544 # Classpath values filled in below (after applying tested_apk_config). 1545 config['javac'] = {} 1546 if options.aar_path: 1547 deps_info['aar_path'] = options.aar_path 1548 if options.unprocessed_jar_path: 1549 deps_info['unprocessed_jar_path'] = options.unprocessed_jar_path 1550 deps_info['interface_jar_path'] = options.interface_jar_path 1551 if options.device_jar_path: 1552 deps_info['device_jar_path'] = options.device_jar_path 1553 if options.host_jar_path: 1554 deps_info['host_jar_path'] = options.host_jar_path 1555 if options.dex_path: 1556 deps_info['dex_path'] = options.dex_path 1557 if is_apk_or_module_target: 1558 all_dex_files.append(options.dex_path) 1559 if options.low_classpath_priority: 1560 deps_info['low_classpath_priority'] = True 1561 if options.type == 'android_apk': 1562 deps_info['apk_path'] = options.apk_path 1563 deps_info['incremental_apk_path'] = options.incremental_apk_path 1564 deps_info['incremental_install_json_path'] = ( 1565 options.incremental_install_json_path) 1566 1567 if options.type == 'android_assets': 1568 all_asset_sources = [] 1569 if options.asset_renaming_sources: 1570 all_asset_sources.extend( 1571 action_helpers.parse_gn_list(options.asset_renaming_sources)) 1572 if options.asset_sources: 1573 all_asset_sources.extend( 1574 action_helpers.parse_gn_list(options.asset_sources)) 1575 1576 deps_info['assets'] = { 1577 'sources': all_asset_sources 1578 } 1579 if options.asset_renaming_destinations: 1580 deps_info['assets']['outputs'] = (action_helpers.parse_gn_list( 1581 options.asset_renaming_destinations)) 1582 if options.disable_asset_compression: 1583 deps_info['assets']['disable_compression'] = True 1584 if options.treat_as_locale_paks: 1585 deps_info['assets']['treat_as_locale_paks'] = True 1586 1587 if options.type == 'android_resources': 1588 deps_info['resources_zip'] = options.resources_zip 1589 if options.resource_overlay: 1590 deps_info['resource_overlay'] = True 1591 if options.srcjar: 1592 deps_info['srcjar'] = options.srcjar 1593 if options.android_manifest: 1594 manifest = AndroidManifest(options.android_manifest) 1595 deps_info['package_name'] = manifest.GetPackageName() 1596 if options.package_name: 1597 deps_info['package_name'] = options.package_name 1598 deps_info['res_sources_path'] = '' 1599 if options.res_sources_path: 1600 deps_info['res_sources_path'] = options.res_sources_path 1601 1602 if (options.requires_android 1603 and options.type == 'java_library') or options.is_robolectric: 1604 if options.package_name: 1605 deps_info['package_name'] = options.package_name 1606 1607 if options.type in ('android_resources', 'android_apk', 'robolectric_binary', 1608 'dist_aar', 'android_app_bundle_module', 'java_library'): 1609 dependency_zips = [] 1610 dependency_zip_overlays = [] 1611 for c in all_resources_deps: 1612 if not c['resources_zip']: 1613 continue 1614 1615 dependency_zips.append(c['resources_zip']) 1616 if c.get('resource_overlay'): 1617 dependency_zip_overlays.append(c['resources_zip']) 1618 1619 extra_package_names = [] 1620 1621 if options.type != 'android_resources': 1622 extra_package_names = [ 1623 c['package_name'] for c in all_resources_deps if 'package_name' in c 1624 ] 1625 if options.package_name: 1626 extra_package_names += [options.package_name] 1627 1628 # android_resources targets which specified recursive_resource_deps may 1629 # have extra_package_names. 1630 for resources_dep in all_resources_deps: 1631 extra_package_names.extend(resources_dep['extra_package_names']) 1632 1633 # In final types (i.e. apks and modules) that create real R.java files, 1634 # they must collect package names from java_libraries as well. 1635 # https://crbug.com/1073476 1636 if options.type != 'java_library': 1637 extra_package_names.extend([ 1638 c['package_name'] for c in all_library_deps if 'package_name' in c 1639 ]) 1640 elif options.recursive_resource_deps: 1641 # Pull extra_package_names from library deps if recursive resource deps 1642 # are required. 1643 extra_package_names = [ 1644 c['package_name'] for c in android_resources_library_deps 1645 if 'package_name' in c 1646 ] 1647 config['deps_info']['includes_recursive_resources'] = True 1648 1649 if options.type in ('dist_aar', 'java_library'): 1650 r_text_files = [ 1651 c['r_text_path'] for c in all_resources_deps if 'r_text_path' in c 1652 ] 1653 deps_info['dependency_r_txt_files'] = r_text_files 1654 1655 if options.type == 'android_apk' and options.tested_apk_config: 1656 config['deps_info']['arsc_package_name'] = ( 1657 tested_apk_config['package_name']) 1658 # We should not shadow the actual R.java files of the apk_under_test by 1659 # creating new R.java files with the same package names in the tested apk. 1660 extra_package_names = [ 1661 package for package in extra_package_names 1662 if package not in tested_apk_config['extra_package_names'] 1663 ] 1664 if options.res_size_info: 1665 config['deps_info']['res_size_info'] = options.res_size_info 1666 1667 # Safe to sort: Build checks that non-overlay resource have no overlap. 1668 dependency_zips.sort() 1669 config['deps_info']['dependency_zips'] = dependency_zips 1670 config['deps_info']['dependency_zip_overlays'] = dependency_zip_overlays 1671 # Order doesn't matter, so make stable. 1672 extra_package_names.sort() 1673 config['deps_info']['extra_package_names'] = extra_package_names 1674 1675 # These are .jars to add to javac classpath but not to runtime classpath. 1676 extra_classpath_jars = action_helpers.parse_gn_list( 1677 options.extra_classpath_jars) 1678 if extra_classpath_jars: 1679 extra_classpath_jars.sort() 1680 deps_info['extra_classpath_jars'] = extra_classpath_jars 1681 1682 mergeable_android_manifests = action_helpers.parse_gn_list( 1683 options.mergeable_android_manifests) 1684 mergeable_android_manifests.sort() 1685 if mergeable_android_manifests: 1686 deps_info['mergeable_android_manifests'] = mergeable_android_manifests 1687 1688 extra_proguard_classpath_jars = [] 1689 proguard_configs = action_helpers.parse_gn_list(options.proguard_configs) 1690 if proguard_configs: 1691 # Make a copy of |proguard_configs| since it's mutated below. 1692 deps_info['proguard_configs'] = list(proguard_configs) 1693 1694 1695 if is_java_target: 1696 classpath_direct_deps = deps.Direct() 1697 classpath_direct_library_deps = deps.Direct('java_library') 1698 1699 # The classpath used to compile this target when annotation processors are 1700 # present. 1701 javac_classpath = set(c['unprocessed_jar_path'] 1702 for c in classpath_direct_library_deps) 1703 # The classpath used to compile this target when annotation processors are 1704 # not present. These are also always used to know when a target needs to be 1705 # rebuilt. 1706 javac_interface_classpath = set(c['interface_jar_path'] 1707 for c in classpath_direct_library_deps) 1708 1709 # Preserve order of |all_library_deps|. Move low priority libraries to the 1710 # end of the classpath. 1711 all_library_deps_sorted_for_classpath = sorted( 1712 all_library_deps[::-1], key=_CompareClasspathPriority) 1713 1714 # The classpath used for bytecode-rewritting. 1715 javac_full_classpath = OrderedSet.fromkeys( 1716 c['unprocessed_jar_path'] 1717 for c in all_library_deps_sorted_for_classpath) 1718 # The classpath used for error prone. 1719 javac_full_interface_classpath = OrderedSet.fromkeys( 1720 c['interface_jar_path'] for c in all_library_deps_sorted_for_classpath) 1721 1722 # Adding base module to classpath to compile against its R.java file 1723 if base_module_build_config: 1724 javac_full_classpath.add( 1725 base_module_build_config['deps_info']['unprocessed_jar_path']) 1726 javac_full_interface_classpath.add( 1727 base_module_build_config['deps_info']['interface_jar_path']) 1728 # Turbine now compiles headers against only the direct classpath, so the 1729 # base module's interface jar must be on the direct interface classpath. 1730 javac_interface_classpath.add( 1731 base_module_build_config['deps_info']['interface_jar_path']) 1732 1733 for dep in classpath_direct_deps: 1734 if 'extra_classpath_jars' in dep: 1735 javac_classpath.update(dep['extra_classpath_jars']) 1736 javac_interface_classpath.update(dep['extra_classpath_jars']) 1737 for dep in all_deps: 1738 if 'extra_classpath_jars' in dep: 1739 javac_full_classpath.update(dep['extra_classpath_jars']) 1740 javac_full_interface_classpath.update(dep['extra_classpath_jars']) 1741 1742 # TODO(agrieve): Might be less confusing to fold these into bootclasspath. 1743 # Deps to add to the compile-time classpath (but not the runtime classpath). 1744 # These are jars specified by input_jars_paths that almost never change. 1745 # Just add them directly to all the classpaths. 1746 if options.extra_classpath_jars: 1747 javac_classpath.update(extra_classpath_jars) 1748 javac_interface_classpath.update(extra_classpath_jars) 1749 javac_full_classpath.update(extra_classpath_jars) 1750 javac_full_interface_classpath.update(extra_classpath_jars) 1751 1752 if is_java_target or options.type == 'android_app_bundle': 1753 # The classpath to use to run this target (or as an input to ProGuard). 1754 device_classpath = [] 1755 # dist_jar configs should not list their device jar in their own classpath 1756 # since the classpath is used to create the device jar itself. 1757 if (is_java_target and options.device_jar_path 1758 and options.type != 'dist_jar'): 1759 device_classpath.append(options.device_jar_path) 1760 device_classpath.extend( 1761 c.get('device_jar_path') for c in all_library_deps 1762 if c.get('device_jar_path')) 1763 if options.type == 'android_app_bundle': 1764 for d in deps.Direct('android_app_bundle_module'): 1765 device_classpath.extend(c for c in d.get('device_classpath', []) 1766 if c not in device_classpath) 1767 1768 all_dist_jar_deps = deps.All('dist_jar') 1769 1770 # We allow lint to be run on android_apk targets, so we collect lint 1771 # artifacts for them. 1772 # We allow lint to be run on android_app_bundle targets, so we need to 1773 # collect lint artifacts for the android_app_bundle_module targets that the 1774 # bundle includes. Different android_app_bundle targets may include different 1775 # android_app_bundle_module targets, so the bundle needs to be able to 1776 # de-duplicate these lint artifacts. 1777 if options.type in ('android_app_bundle_module', 'android_apk'): 1778 # Collect all sources and resources at the apk/bundle_module level. 1779 lint_aars = set() 1780 lint_srcjars = set() 1781 lint_sources = set() 1782 lint_resource_sources = set() 1783 lint_resource_zips = set() 1784 1785 if options.target_sources_file: 1786 lint_sources.add(options.target_sources_file) 1787 if options.bundled_srcjars: 1788 lint_srcjars.update(deps_info['bundled_srcjars']) 1789 for c in all_library_deps: 1790 if c['chromium_code'] and c['requires_android']: 1791 if 'target_sources_file' in c: 1792 lint_sources.add(c['target_sources_file']) 1793 lint_srcjars.update(c['bundled_srcjars']) 1794 if 'aar_path' in c: 1795 lint_aars.add(c['aar_path']) 1796 1797 if options.res_sources_path: 1798 lint_resource_sources.add(options.res_sources_path) 1799 if options.resources_zip: 1800 lint_resource_zips.add(options.resources_zip) 1801 for c in all_resources_deps: 1802 if c['chromium_code']: 1803 # Prefer res_sources_path to resources_zips so that lint errors have 1804 # real paths and to avoid needing to extract during lint. 1805 if c['res_sources_path']: 1806 lint_resource_sources.add(c['res_sources_path']) 1807 else: 1808 lint_resource_zips.add(c['resources_zip']) 1809 1810 deps_info['lint_aars'] = sorted(lint_aars) 1811 deps_info['lint_srcjars'] = sorted(lint_srcjars) 1812 deps_info['lint_sources'] = sorted(lint_sources) 1813 deps_info['lint_resource_sources'] = sorted(lint_resource_sources) 1814 deps_info['lint_resource_zips'] = sorted(lint_resource_zips) 1815 deps_info['lint_extra_android_manifests'] = [] 1816 1817 if options.type == 'android_apk': 1818 assert options.android_manifest, 'Android APKs must define a manifest' 1819 deps_info['lint_android_manifest'] = options.android_manifest 1820 1821 if options.type == 'android_app_bundle': 1822 module_config_paths = action_helpers.parse_gn_list( 1823 options.module_build_configs) 1824 module_configs = [GetDepConfig(c) for c in module_config_paths] 1825 module_configs_by_name = {d['module_name']: d for d in module_configs} 1826 per_module_fields = [ 1827 'device_classpath', 'trace_event_rewritten_device_classpath', 1828 'all_dex_files' 1829 ] 1830 lint_aars = set() 1831 lint_srcjars = set() 1832 lint_sources = set() 1833 lint_resource_sources = set() 1834 lint_resource_zips = set() 1835 lint_extra_android_manifests = set() 1836 config['modules'] = {} 1837 modules = config['modules'] 1838 for n, c in module_configs_by_name.items(): 1839 if n == 'base': 1840 assert 'base_module_config' not in deps_info, ( 1841 'Must have exactly 1 base module!') 1842 deps_info['package_name'] = c['package_name'] 1843 deps_info['version_code'] = c['version_code'] 1844 deps_info['version_name'] = c['version_name'] 1845 deps_info['base_module_config'] = c['path'] 1846 deps_info['lint_android_manifest'] = c['android_manifest'] 1847 else: 1848 # All manifests nodes are merged into the main manfiest by lint.py. 1849 lint_extra_android_manifests.add(c['android_manifest']) 1850 1851 lint_extra_android_manifests.update(c['extra_android_manifests']) 1852 lint_aars.update(c['lint_aars']) 1853 lint_srcjars.update(c['lint_srcjars']) 1854 lint_sources.update(c['lint_sources']) 1855 lint_resource_sources.update(c['lint_resource_sources']) 1856 lint_resource_zips.update(c['lint_resource_zips']) 1857 module = modules[n] = {} 1858 for f in per_module_fields: 1859 if f in c: 1860 module[f] = c[f] 1861 deps_info['lint_aars'] = sorted(lint_aars) 1862 deps_info['lint_srcjars'] = sorted(lint_srcjars) 1863 deps_info['lint_sources'] = sorted(lint_sources) 1864 deps_info['lint_resource_sources'] = sorted(lint_resource_sources) 1865 deps_info['lint_resource_zips'] = sorted(lint_resource_zips) 1866 deps_info['lint_extra_android_manifests'] = sorted( 1867 lint_extra_android_manifests) 1868 1869 _DedupFeatureModuleSharedCode(options.uses_split, modules, 1870 per_module_fields) 1871 1872 system_jars = [c['unprocessed_jar_path'] for c in system_library_deps] 1873 system_interface_jars = [c['interface_jar_path'] for c in system_library_deps] 1874 if system_library_deps: 1875 config['android'] = {} 1876 config['android']['sdk_interface_jars'] = system_interface_jars 1877 config['android']['sdk_jars'] = system_jars 1878 1879 if options.type in ('android_apk', 'dist_aar', 1880 'dist_jar', 'android_app_bundle_module', 'android_app_bundle'): 1881 for c in all_deps: 1882 proguard_configs.extend(c.get('proguard_configs', [])) 1883 extra_proguard_classpath_jars.extend(c.get('extra_classpath_jars', [])) 1884 if options.type == 'android_app_bundle': 1885 for c in deps.Direct('android_app_bundle_module'): 1886 proguard_configs.extend(p for p in c.get('proguard_configs', [])) 1887 if options.type == 'android_app_bundle': 1888 for d in deps.Direct('android_app_bundle_module'): 1889 extra_proguard_classpath_jars.extend( 1890 c for c in d.get('proguard_classpath_jars', []) 1891 if c not in extra_proguard_classpath_jars) 1892 1893 if options.type == 'android_app_bundle': 1894 deps_proguard_enabled = [] 1895 deps_proguard_disabled = [] 1896 for d in deps.Direct('android_app_bundle_module'): 1897 if not d['device_classpath']: 1898 # We don't care about modules that have no Java code for proguarding. 1899 continue 1900 if d['proguard_enabled']: 1901 deps_proguard_enabled.append(d['name']) 1902 else: 1903 deps_proguard_disabled.append(d['name']) 1904 if deps_proguard_enabled and deps_proguard_disabled: 1905 raise Exception('Deps %s have proguard enabled while deps %s have ' 1906 'proguard disabled' % (deps_proguard_enabled, 1907 deps_proguard_disabled)) 1908 deps_info['proguard_enabled'] = bool(options.proguard_enabled) 1909 1910 if options.proguard_mapping_path: 1911 deps_info['proguard_mapping_path'] = options.proguard_mapping_path 1912 1913 # The java code for an instrumentation test apk is assembled differently for 1914 # ProGuard vs. non-ProGuard. 1915 # 1916 # Without ProGuard: Each library's jar is dexed separately and then combined 1917 # into a single classes.dex. A test apk will include all dex files not already 1918 # present in the apk-under-test. At runtime all test code lives in the test 1919 # apk, and the program code lives in the apk-under-test. 1920 # 1921 # With ProGuard: Each library's .jar file is fed into ProGuard, which outputs 1922 # a single .jar, which is then dexed into a classes.dex. A test apk includes 1923 # all jar files from the program and the tests because having them separate 1924 # doesn't work with ProGuard's whole-program optimizations. Although the 1925 # apk-under-test still has all of its code in its classes.dex, none of it is 1926 # used at runtime because the copy of it within the test apk takes precidence. 1927 1928 if options.type == 'android_apk' and options.tested_apk_config: 1929 if tested_apk_config['proguard_enabled']: 1930 assert options.proguard_enabled, ('proguard must be enabled for ' 1931 'instrumentation apks if it\'s enabled for the tested apk.') 1932 # Mutating lists, so no need to explicitly re-assign to dict. 1933 proguard_configs.extend( 1934 p for p in tested_apk_config['proguard_all_configs']) 1935 extra_proguard_classpath_jars.extend( 1936 p for p in tested_apk_config['proguard_classpath_jars']) 1937 tested_apk_config = GetDepConfig(options.tested_apk_config) 1938 deps_info['proguard_under_test_mapping'] = ( 1939 tested_apk_config['proguard_mapping_path']) 1940 elif options.proguard_enabled: 1941 # Not sure why you'd want to proguard the test apk when the under-test apk 1942 # is not proguarded, but it's easy enough to support. 1943 deps_info['proguard_under_test_mapping'] = '' 1944 1945 # Add all tested classes to the test's classpath to ensure that the test's 1946 # java code is a superset of the tested apk's java code 1947 device_classpath_extended = list(device_classpath) 1948 device_classpath_extended.extend( 1949 p for p in tested_apk_config['device_classpath'] 1950 if p not in device_classpath) 1951 # Include in the classpath classes that are added directly to the apk under 1952 # test (those that are not a part of a java_library). 1953 javac_classpath.add(tested_apk_config['unprocessed_jar_path']) 1954 javac_interface_classpath.add(tested_apk_config['interface_jar_path']) 1955 javac_full_classpath.add(tested_apk_config['unprocessed_jar_path']) 1956 javac_full_interface_classpath.add(tested_apk_config['interface_jar_path']) 1957 javac_full_classpath.update(tested_apk_config['javac_full_classpath']) 1958 javac_full_interface_classpath.update( 1959 tested_apk_config['javac_full_interface_classpath']) 1960 1961 # Exclude .jar files from the test apk that exist within the apk under test. 1962 tested_apk_library_deps = tested_apk_deps.All('java_library') 1963 tested_apk_dex_files = {c['dex_path'] for c in tested_apk_library_deps} 1964 all_dex_files = [p for p in all_dex_files if p not in tested_apk_dex_files] 1965 tested_apk_jar_files = set(tested_apk_config['device_classpath']) 1966 device_classpath = [ 1967 p for p in device_classpath if p not in tested_apk_jar_files 1968 ] 1969 1970 if options.type in ('android_apk', 'dist_aar', 'dist_jar', 1971 'android_app_bundle_module', 'android_app_bundle'): 1972 deps_info['proguard_all_configs'] = sorted(set(proguard_configs)) 1973 deps_info['proguard_classpath_jars'] = sorted( 1974 set(extra_proguard_classpath_jars)) 1975 1976 if options.type in ('dist_jar', 'java_binary', 'robolectric_binary'): 1977 # The classpath to use to run this target. 1978 host_classpath = [] 1979 if options.host_jar_path: 1980 host_classpath.append(options.host_jar_path) 1981 host_classpath.extend(c['host_jar_path'] for c in all_library_deps) 1982 # Collect all the dist_jar host jars. 1983 dist_jar_host_jars = [ 1984 c['host_jar_path'] for c in all_dist_jar_deps if 'host_jar_path' in c 1985 ] 1986 # Collect all the jars that went into the dist_jar host jars. 1987 dist_jar_host_classpath = set() 1988 for c in all_dist_jar_deps: 1989 dist_jar_host_classpath.update(c['host_classpath']) 1990 # Remove the jars that went into the dist_jar host jars. 1991 host_classpath = [ 1992 p for p in host_classpath if p not in dist_jar_host_classpath 1993 ] 1994 # Add the dist_jar host jars themselves instead. 1995 host_classpath += dist_jar_host_jars 1996 deps_info['host_classpath'] = host_classpath 1997 1998 if is_java_target: 1999 2000 def _CollectListsFromDeps(deps, key_name): 2001 combined = set() 2002 for config in deps: 2003 combined.update(config.get(key_name, [])) 2004 return combined 2005 2006 dist_jar_device_classpath = _CollectListsFromDeps(all_dist_jar_deps, 2007 'device_classpath') 2008 dist_jar_javac_full_classpath = _CollectListsFromDeps( 2009 all_dist_jar_deps, 'javac_full_classpath') 2010 dist_jar_javac_full_interface_classpath = _CollectListsFromDeps( 2011 all_dist_jar_deps, 'javac_full_interface_classpath') 2012 dist_jar_child_dex_files = _CollectListsFromDeps(all_dist_jar_deps, 2013 'all_dex_files') 2014 2015 def _CollectSinglesFromDeps(deps, key_name): 2016 return [config[key_name] for config in deps if key_name in config] 2017 2018 dist_jar_device_jars = _CollectSinglesFromDeps(all_dist_jar_deps, 2019 'device_jar_path') 2020 dist_jar_combined_dex_files = _CollectSinglesFromDeps( 2021 all_dist_jar_deps, 'dex_path') 2022 dist_jar_interface_jars = _CollectSinglesFromDeps(all_dist_jar_deps, 2023 'interface_jar_path') 2024 dist_jar_unprocessed_jars = _CollectSinglesFromDeps(all_dist_jar_deps, 2025 'unprocessed_jar_path') 2026 2027 device_classpath = [ 2028 p for p in device_classpath if p not in dist_jar_device_classpath 2029 ] 2030 device_classpath += dist_jar_device_jars 2031 2032 javac_full_classpath.difference_update(dist_jar_javac_full_classpath) 2033 javac_full_classpath.update(dist_jar_unprocessed_jars) 2034 2035 javac_full_interface_classpath.difference_update( 2036 dist_jar_javac_full_interface_classpath) 2037 javac_full_interface_classpath.update(dist_jar_interface_jars) 2038 2039 javac_interface_classpath.update(dist_jar_interface_jars) 2040 javac_classpath.update(dist_jar_unprocessed_jars) 2041 2042 if is_apk_or_module_target or options.type == 'dist_jar': 2043 all_dex_files = [ 2044 p for p in all_dex_files if p not in dist_jar_child_dex_files 2045 ] 2046 all_dex_files += dist_jar_combined_dex_files 2047 2048 if options.final_dex_path: 2049 config['final_dex'] = {'path': options.final_dex_path} 2050 if is_apk_or_module_target or options.type == 'dist_jar': 2051 # Dependencies for the final dex file of an apk. 2052 deps_info['all_dex_files'] = all_dex_files 2053 2054 if is_java_target: 2055 config['javac']['classpath'] = sorted(javac_classpath) 2056 config['javac']['interface_classpath'] = sorted(javac_interface_classpath) 2057 # Direct() will be of type 'java_annotation_processor', and so not included 2058 # in All('java_library'). 2059 # Annotation processors run as part of the build, so need host_jar_path. 2060 config['javac']['processor_classpath'] = [ 2061 c['host_jar_path'] for c in processor_deps.Direct() 2062 if c.get('host_jar_path') 2063 ] 2064 config['javac']['processor_classpath'] += [ 2065 c['host_jar_path'] for c in processor_deps.All('java_library') 2066 ] 2067 config['javac']['processor_classes'] = sorted( 2068 c['main_class'] for c in processor_deps.Direct()) 2069 deps_info['javac_full_classpath'] = list(javac_full_classpath) 2070 deps_info['javac_full_interface_classpath'] = list( 2071 javac_full_interface_classpath) 2072 elif options.type == 'android_app_bundle': 2073 # bundles require javac_full_classpath to create .aab.jar.info and require 2074 # javac_full_interface_classpath for lint. 2075 javac_full_classpath = OrderedSet() 2076 javac_full_interface_classpath = OrderedSet() 2077 for d in deps.Direct('android_app_bundle_module'): 2078 javac_full_classpath.update(d['javac_full_classpath']) 2079 javac_full_interface_classpath.update(d['javac_full_interface_classpath']) 2080 javac_full_classpath.add(d['unprocessed_jar_path']) 2081 javac_full_interface_classpath.add(d['interface_jar_path']) 2082 deps_info['javac_full_classpath'] = list(javac_full_classpath) 2083 deps_info['javac_full_interface_classpath'] = list( 2084 javac_full_interface_classpath) 2085 2086 if options.type in ('android_apk', 'android_app_bundle', 2087 'android_app_bundle_module', 'dist_aar', 'dist_jar'): 2088 deps_info['device_classpath'] = device_classpath 2089 if options.trace_events_jar_dir: 2090 trace_event_rewritten_device_classpath = [] 2091 for jar_path in device_classpath: 2092 file_path = jar_path.replace('../', '') 2093 file_path = file_path.replace('obj/', '') 2094 file_path = file_path.replace('gen/', '') 2095 file_path = file_path.replace('.jar', '.tracing_rewritten.jar') 2096 rewritten_jar_path = os.path.join(options.trace_events_jar_dir, 2097 file_path) 2098 trace_event_rewritten_device_classpath.append(rewritten_jar_path) 2099 2100 deps_info['trace_event_rewritten_device_classpath'] = ( 2101 trace_event_rewritten_device_classpath) 2102 2103 if options.tested_apk_config: 2104 deps_info['device_classpath_extended'] = device_classpath_extended 2105 2106 if options.type in ('android_apk', 'dist_jar'): 2107 all_interface_jars = [] 2108 if options.interface_jar_path: 2109 all_interface_jars.append(options.interface_jar_path) 2110 all_interface_jars.extend(c['interface_jar_path'] for c in all_library_deps) 2111 2112 config['dist_jar'] = { 2113 'all_interface_jars': all_interface_jars, 2114 } 2115 2116 if is_apk_or_module_target: 2117 manifest = AndroidManifest(options.android_manifest) 2118 deps_info['package_name'] = manifest.GetPackageName() 2119 if not options.tested_apk_config and manifest.GetInstrumentationElements(): 2120 # This must then have instrumentation only for itself. 2121 manifest.CheckInstrumentationElements(manifest.GetPackageName()) 2122 2123 library_paths = [] 2124 java_libraries_list = None 2125 if options.shared_libraries_runtime_deps: 2126 library_paths = _ExtractSharedLibsFromRuntimeDeps( 2127 options.shared_libraries_runtime_deps) 2128 java_libraries_list = _CreateJavaLibrariesList(library_paths) 2129 all_inputs.append(options.shared_libraries_runtime_deps) 2130 2131 secondary_abi_library_paths = [] 2132 if options.secondary_abi_shared_libraries_runtime_deps: 2133 secondary_abi_library_paths = _ExtractSharedLibsFromRuntimeDeps( 2134 options.secondary_abi_shared_libraries_runtime_deps) 2135 secondary_abi_library_paths.sort() 2136 paths_without_parent_dirs = [ 2137 p for p in secondary_abi_library_paths if os.path.sep not in p 2138 ] 2139 if paths_without_parent_dirs: 2140 sys.stderr.write('Found secondary native libraries from primary ' 2141 'toolchain directory. This is a bug!\n') 2142 sys.stderr.write('\n'.join(paths_without_parent_dirs)) 2143 sys.stderr.write('\n\nIt may be helpful to run: \n') 2144 sys.stderr.write(' gn path out/Default //chrome/android:' 2145 'monochrome_secondary_abi_lib //base:base\n') 2146 sys.exit(1) 2147 2148 all_inputs.append(options.secondary_abi_shared_libraries_runtime_deps) 2149 2150 native_library_placeholder_paths = action_helpers.parse_gn_list( 2151 options.native_lib_placeholders) 2152 native_library_placeholder_paths.sort() 2153 2154 secondary_native_library_placeholder_paths = action_helpers.parse_gn_list( 2155 options.secondary_native_lib_placeholders) 2156 secondary_native_library_placeholder_paths.sort() 2157 2158 loadable_modules = action_helpers.parse_gn_list(options.loadable_modules) 2159 loadable_modules.sort() 2160 secondary_abi_loadable_modules = action_helpers.parse_gn_list( 2161 options.secondary_abi_loadable_modules) 2162 secondary_abi_loadable_modules.sort() 2163 2164 config['native'] = { 2165 'libraries': 2166 library_paths, 2167 'native_library_placeholders': 2168 native_library_placeholder_paths, 2169 'secondary_abi_libraries': 2170 secondary_abi_library_paths, 2171 'secondary_native_library_placeholders': 2172 secondary_native_library_placeholder_paths, 2173 'java_libraries_list': 2174 java_libraries_list, 2175 'library_always_compress': 2176 options.library_always_compress, 2177 'loadable_modules': 2178 loadable_modules, 2179 'secondary_abi_loadable_modules': 2180 secondary_abi_loadable_modules, 2181 } 2182 2183 # Collect java resources 2184 java_resources_jars = [d['java_resources_jar'] for d in all_library_deps 2185 if 'java_resources_jar' in d] 2186 if options.tested_apk_config: 2187 tested_apk_resource_jars = [d['java_resources_jar'] 2188 for d in tested_apk_library_deps 2189 if 'java_resources_jar' in d] 2190 java_resources_jars = [jar for jar in java_resources_jars 2191 if jar not in tested_apk_resource_jars] 2192 java_resources_jars.sort() 2193 config['java_resources_jars'] = java_resources_jars 2194 2195 if is_apk_or_module_target or options.type == 'robolectric_binary': 2196 # android_resources deps which had recursive_resource_deps set should not 2197 # have the manifests from the recursively collected deps added to this 2198 # module. This keeps the manifest declarations in the child DFMs, since they 2199 # will have the Java implementations. 2200 def ExcludeRecursiveResourcesDeps(config): 2201 return not config.get('includes_recursive_resources', False) 2202 2203 extra_manifest_deps = [ 2204 GetDepConfig(p) for p in GetAllDepsConfigsInOrder( 2205 deps_configs_paths, filter_func=ExcludeRecursiveResourcesDeps) 2206 ] 2207 # Manifests are listed from highest priority to lowest priority. 2208 # Ensure directly manfifests come first, and then sort the rest by name. 2209 # https://developer.android.com/build/manage-manifests#merge_priorities 2210 deps_info['extra_android_manifests'] = list(mergeable_android_manifests) 2211 manifests_from_deps = [] 2212 for c in extra_manifest_deps: 2213 manifests_from_deps += c.get('mergeable_android_manifests', []) 2214 manifests_from_deps.sort(key=lambda p: (os.path.basename(p), p)) 2215 deps_info['extra_android_manifests'] += manifests_from_deps 2216 2217 assets, uncompressed_assets, locale_paks = _MergeAssets( 2218 deps.All('android_assets')) 2219 deps_info['assets'] = assets 2220 deps_info['uncompressed_assets'] = uncompressed_assets 2221 deps_info['locales_java_list'] = _CreateJavaLocaleListFromAssets( 2222 uncompressed_assets, locale_paks) 2223 if options.suffix_apk_assets_used_by: 2224 if options.suffix_apk_assets_used_by == options.build_config: 2225 target_config = deps_info 2226 else: 2227 target_config = GetDepConfig(options.suffix_apk_assets_used_by) 2228 all_assets = (target_config['assets'] + 2229 target_config['uncompressed_assets']) 2230 suffix = '+' + target_config['package_name'] + '+' 2231 suffix_names = { 2232 x.split(':', 1)[1].replace(suffix, '') 2233 for x in all_assets 2234 } 2235 deps_info['assets'] = _SuffixAssets(suffix_names, suffix, assets) 2236 deps_info['uncompressed_assets'] = _SuffixAssets(suffix_names, suffix, 2237 uncompressed_assets) 2238 config['apk_assets_suffixed_list'] = ','.join( 2239 f'"assets/{x}"' for x in sorted(suffix_names)) 2240 config['apk_assets_suffix'] = suffix 2241 2242 if options.java_resources_jar_path: 2243 deps_info['java_resources_jar'] = options.java_resources_jar_path 2244 2245 # DYNAMIC FEATURE MODULES: 2246 # There are two approaches to dealing with modules dependencies: 2247 # 1) Perform steps in android_apk_or_module(), with only the knowledge of 2248 # ancesstor splits. Our implementation currently allows only for 2 levels: 2249 # base -> parent -> leaf 2250 # Bundletool normally fails if two leaf nodes merge the same manifest or 2251 # resources. The fix is to add the common dep to the chrome or base module 2252 # so that our deduplication logic will work. 2253 # RemoveObjDups() implements this approach. 2254 # 2) Perform steps in android_app_bundle(), with knowledge of full set of 2255 # modules. This is required for dex because it can handle the case of two 2256 # leaf nodes having the same dep, and promoting that dep to their common 2257 # parent. 2258 # _DedupFeatureModuleSharedCode() implements this approach. 2259 if base_module_build_config: 2260 ancestors = [base_module_build_config] 2261 if parent_module_build_config is not base_module_build_config: 2262 ancestors += [parent_module_build_config] 2263 for ancestor in ancestors: 2264 RemoveObjDups(config, ancestor, 'deps_info', 'dependency_zips') 2265 RemoveObjDups(config, ancestor, 'deps_info', 'dependency_zip_overlays') 2266 RemoveObjDups(config, ancestor, 'deps_info', 'extra_android_manifests') 2267 RemoveObjDups(config, ancestor, 'deps_info', 'extra_package_names') 2268 2269 if is_java_target: 2270 jar_to_target = {} 2271 _AddJarMapping(jar_to_target, [deps_info]) 2272 _AddJarMapping(jar_to_target, all_deps) 2273 if base_module_build_config: 2274 _AddJarMapping(jar_to_target, [base_module_build_config['deps_info']]) 2275 if parent_module_build_config is not base_module_build_config: 2276 _AddJarMapping(jar_to_target, [parent_module_build_config['deps_info']]) 2277 if options.tested_apk_config: 2278 _AddJarMapping(jar_to_target, [tested_apk_config]) 2279 for jar, target in zip(tested_apk_config['javac_full_classpath'], 2280 tested_apk_config['javac_full_classpath_targets']): 2281 jar_to_target[jar] = target 2282 2283 # Used by bytecode_processor to give better error message when missing 2284 # deps are found. Both javac_full_classpath_targets and javac_full_classpath 2285 # must be in identical orders, as they get passed as separate arrays and 2286 # then paired up based on index. 2287 config['deps_info']['javac_full_classpath_targets'] = [ 2288 jar_to_target[x] for x in deps_info['javac_full_classpath'] 2289 ] 2290 2291 build_utils.WriteJson(config, options.build_config, only_if_changed=True) 2292 2293 if options.depfile: 2294 action_helpers.write_depfile(options.depfile, options.build_config, 2295 sorted(set(all_inputs))) 2296 2297 if options.store_deps_for_debugging_to: 2298 GetDepConfig(options.build_config) # Add it to cache. 2299 _CopyBuildConfigsForDebugging(options.store_deps_for_debugging_to) 2300 2301 return 0 2302 2303 2304if __name__ == '__main__': 2305 sys.exit(main(sys.argv[1:])) 2306