1#!/usr/bin/env python3 2# 3# Copyright 2014 The Chromium Authors 4# Use of this source code is governed by a BSD-style license that can be 5# found in the LICENSE file. 6 7"""Writes a build_config file. 8 9The build_config file for a target is a json file containing information about 10how to build that target based on the target's dependencies. This includes 11things like: the javac classpath, the list of android resources dependencies, 12etc. It also includes the information needed to create the build_config for 13other targets that depend on that one. 14 15Android build scripts should not refer to the build_config directly, and the 16build specification should instead pass information in using the special 17file-arg syntax (see build_utils.py:ExpandFileArgs). That syntax allows passing 18of values in a json dict in a file and looks like this: 19 --python-arg=@FileArg(build_config_path:javac:classpath) 20 21Note: If paths to input files are passed in this way, it is important that: 22 1. inputs/deps of the action ensure that the files are available the first 23 time the action runs. 24 2. Either (a) or (b) 25 a. inputs/deps ensure that the action runs whenever one of the files changes 26 b. the files are added to the action's depfile 27 28NOTE: All paths within .build_config files are relative to $OUTPUT_CHROMIUM_DIR. 29 30This is a technical note describing the format of .build_config files. 31Please keep it updated when changing this script. For extraction and 32visualization instructions, see build/android/docs/build_config.md 33 34------------- BEGIN_MARKDOWN --------------------------------------------------- 35The .build_config file format 36=== 37 38# Introduction 39 40This document tries to explain the format of `.build_config` generated during 41the Android build of Chromium. For a higher-level explanation of these files, 42please read 43[build/android/docs/build_config.md](build/android/docs/build_config.md). 44 45# The `deps_info` top-level dictionary: 46 47All `.build_config` files have a required `'deps_info'` key, whose value is a 48dictionary describing the target and its dependencies. The latter has the 49following required keys: 50 51## Required keys in `deps_info`: 52 53* `deps_info['type']`: The target type as a string. 54 55 The following types are known by the internal GN build rules and the 56 build scripts altogether: 57 58 * [java_binary](#target_java_binary) 59 * [java_annotation_processor](#target_java_annotation_processor) 60 * [robolectric_binary](#target_robolectric_binary) 61 * [java_library](#target_java_library) 62 * [android_assets](#target_android_assets) 63 * [android_resources](#target_android_resources) 64 * [android_apk](#target_android_apk) 65 * [android_app_bundle_module](#target_android_app_bundle_module) 66 * [android_app_bundle](#target_android_app_bundle) 67 * [dist_jar](#target_dist_jar) 68 * [dist_aar](#target_dist_aar) 69 * [group](#target_group) 70 71 See later sections for more details of some of these. 72 73* `deps_info['path']`: Path to the target's `.build_config` file. 74 75* `deps_info['name']`: Nothing more than the basename of `deps_info['path']` 76at the moment. 77 78* `deps_info['deps_configs']`: List of paths to the `.build_config` files of 79all *direct* dependencies of the current target. 80 81 NOTE: Because the `.build_config` of a given target is always generated 82 after the `.build_config` of its dependencies, the `write_build_config.py` 83 script can use chains of `deps_configs` to compute transitive dependencies 84 for each target when needed. 85 86## Optional keys in `deps_info`: 87 88The following keys will only appear in the `.build_config` files of certain 89target types: 90 91* `deps_info['requires_android']`: True to indicate that the corresponding 92code uses Android-specific APIs, and thus cannot run on the host within a 93regular JVM. May only appear in Java-related targets. 94 95* `deps_info['supports_android']`: 96May appear in Java-related targets, and indicates that 97the corresponding code doesn't use Java APIs that are not available on 98Android. As such it may run either on the host or on an Android device. 99 100* `deps_info['assets']`: 101Only seen for the [`android_assets`](#target_android_assets) type. See below. 102 103* `deps_info['package_name']`: Java package name associated with this target. 104 105 NOTE: For `android_resources` targets, 106 this is the package name for the corresponding R class. For `android_apk` 107 targets, this is the corresponding package name. This does *not* appear for 108 other target types. 109 110* `deps_info['android_manifest']`: 111Path to an AndroidManifest.xml file related to the current target. 112 113* `deps_info['base_module_config']`: 114Only seen for the [`android_app_bundle`](#target_android_app_bundle) type. 115Path to the base module for the bundle. 116 117* `deps_info['module_name']`: 118Only seen for the 119[`android_app_bundle_module`](#target_android_app_bundle_module) 120type. The name of the feature module. 121 122* `deps_info['dependency_zips']`: 123List of `deps_info['resources_zip']` entries for all `android_resources` 124dependencies for the current target. 125 126* `deps_info['extra_package_names']`: 127Always empty for `android_resources` types. Otherwise, 128the list of `deps_info['package_name']` entries for all `android_resources` 129dependencies for the current target. Computed automatically by 130`write_build_config.py`. 131 132* `deps_info['dependency_r_txt_files']`: 133Exists only on dist_aar. It is the list of deps_info['r_text_path'] from 134transitive dependencies. Computed automatically. 135 136 137# `.build_config` target types description: 138 139## <a name="target_group">Target type `group`</a>: 140 141This type corresponds to a simple target that is only used to group 142dependencies. It matches the `java_group()` GN template. Its only top-level 143`deps_info` keys are `supports_android` (always True), and `deps_configs`. 144 145 146## <a name="target_android_resources">Target type `android_resources`</a>: 147 148This type corresponds to targets that are used to group Android resource files. 149For example, all `android_resources` dependencies of an `android_apk` will 150end up packaged into the final APK by the build system. 151 152It uses the following keys: 153 154 155* `deps_info['res_sources_path']`: 156Path to file containing a list of resource source files used by the 157android_resources target. 158 159* `deps_info['resources_zip']`: 160*Required*. Path to the `.resources.zip` file that contains all raw/uncompiled 161resource files for this target (and also no `R.txt`, `R.java` or `R.class`). 162 163 If `deps_info['res_sources_path']` is missing, this must point to a prebuilt 164 `.aar` archive containing resources. Otherwise, this will point to a zip 165 archive generated at build time, wrapping the sources listed in 166 `deps_info['res_sources_path']` into a single zip file. 167 168* `deps_info['package_name']`: 169Java package name that the R class for this target belongs to. 170 171* `deps_info['android_manifest']`: 172Optional. Path to the top-level Android manifest file associated with these 173resources (if not provided, an empty manifest will be used to generate R.txt). 174 175* `deps_info['resource_overlay']`: 176Optional. Whether the resources in resources_zip should override resources with 177the same name. Does not affect the behaviour of any android_resources() 178dependencies of this target. If a target with resource_overlay=true depends 179on another target with resource_overlay=true the target with the dependency 180overrides the other. 181 182* `deps_info['r_text_path']`: 183Provide the path to the `R.txt` file that describes the resources wrapped by 184this target. Normally this file is generated from the content of the resource 185directories or zip file, but some targets can provide their own `R.txt` file 186if they want. 187 188* `deps_info['srcjar_path']`: 189Path to the `.srcjar` file that contains the auto-generated `R.java` source 190file corresponding to the content of `deps_info['r_text_path']`. This is 191*always* generated from the content of `deps_info['r_text_path']` by the 192`build/android/gyp/process_resources.py` script. 193 194## <a name="target_android_assets">Target type `android_assets`</a>: 195 196This type corresponds to targets used to group Android assets, i.e. liberal 197files that will be placed under `//assets/` within the final APK. 198 199These use an `deps_info['assets']` key to hold a dictionary of values related 200to assets covered by this target. 201 202* `assets['sources']`: 203The list of all asset source paths for this target. Each source path can 204use an optional `:<zipPath>` suffix, where `<zipPath>` is the final location 205of the assets (relative to `//assets/`) within the APK. 206 207* `assets['outputs']`: 208Optional. Some of the sources might be renamed before being stored in the 209final //assets/ sub-directory. When this happens, this contains a list of 210all renamed output file paths 211 212 NOTE: When not empty, the first items of `assets['sources']` must match 213 every item in this list. Extra sources correspond to non-renamed sources. 214 215 NOTE: This comes from the `asset_renaming_destinations` parameter for the 216 `android_assets()` GN template. 217 218* `assets['disable_compression']`: 219Optional. Will be True to indicate that these assets should be stored 220uncompressed in the final APK. For example, this is necessary for locale 221.pak files used by the System WebView feature. 222 223* `assets['treat_as_locale_paks']`: 224Optional. Will be True to indicate that these assets are locale `.pak` files 225(containing localized strings for C++). These are later processed to generate 226a special ``.build_config`.java` source file, listing all supported Locales in 227the current build. 228 229 230## <a name="target_java_library">Target type `java_library`</a>: 231 232This type is used to describe target that wrap Java bytecode, either created 233by compiling sources, or providing them with a prebuilt jar. 234 235* `deps_info['public_deps_configs']`: List of paths to the `.build_config` files 236of *direct* dependencies of the current target which are exposed as part of the 237current target's public API. 238 239* `deps_info['unprocessed_jar_path']`: 240Path to the original .jar file for this target, before any kind of processing 241through Proguard or other tools. For most targets this is generated 242from sources, with a name like `$target_name.javac.jar`. However, when using 243a prebuilt jar, this will point to the source archive directly. 244 245* `deps_info['device_jar_path']`: 246Path to a file that is the result of processing 247`deps_info['unprocessed_jar_path']` with various tools (ready to be dexed). 248 249* `deps_info['host_jar_path']`: 250Path to a file that is the result of processing 251`deps_info['unprocessed_jar_path']` with various tools (use by java_binary). 252 253* `deps_info['interface_jar_path']: 254Path to the interface jar generated for this library. This corresponds to 255a jar file that only contains declarations. Generated by running the `ijar` on 256`deps_info['unprocessed_jar_path']` or the `turbine` tool on source files. 257 258* `deps_info['dex_path']`: 259Path to the `.dex` file generated for this target, from 260`deps_info['device_jar_path']` unless this comes from a prebuilt `.aar` archive. 261 262* `deps_info['is_prebuilt']`: 263True to indicate that this target corresponds to a prebuilt `.jar` file. 264In this case, `deps_info['unprocessed_jar_path']` will point to the source 265`.jar` file. Otherwise, it will be point to a build-generated file. 266 267* `deps_info['target_sources_file']`: 268Path to a single `.sources` file listing all the Java and Kotlin sources that 269were used to generate the library (simple text format, one `.jar` path per 270line). 271 272* `deps_info['lint_android_manifest']`: 273Path to an AndroidManifest.xml file to use for this lint target. 274 275* `deps_info['lint_sources']`: 276The list of all `deps_info['target_sources_file']` entries for all library 277dependencies that are chromium code. Note: this is a list of files, where each 278file contains a list of Java and Kotlin source files. This is used for lint. 279 280* `deps_info['lint_aars']`: 281List of all aars from transitive java dependencies. This allows lint to collect 282their custom annotations.zip and run checks like @IntDef on their annotations. 283 284* `deps_info['lint_srcjars']`: 285List of all bundled srcjars of all transitive java library targets. Excludes 286non-chromium java libraries. 287 288* `deps_info['lint_resource_sources']`: 289List of all resource sources files belonging to all transitive resource 290dependencies of this target. Excludes resources owned by non-chromium code. 291 292* `deps_info['lint_resource_zips']`: 293List of all resource zip files belonging to all transitive resource dependencies 294of this target. Excludes resources owned by non-chromium code. 295 296* `deps_info['javac']`: 297A dictionary containing information about the way the sources in this library 298are compiled. Appears also on other Java-related targets. See the [dedicated 299section about this](#dict_javac) below for details. 300 301* `deps_info['javac_full_classpath']`: 302The classpath used when performing bytecode processing. Essentially the 303collection of all `deps_info['unprocessed_jar_path']` entries for the target 304and all its dependencies. 305 306* `deps_info['javac_full_interface_classpath']`: 307The classpath used when using the errorprone compiler. 308 309* `deps_info['proguard_enabled"]`: 310True to indicate that ProGuard processing is enabled for this target. 311 312* `deps_info['proguard_configs"]`: 313A list of paths to ProGuard configuration files related to this library. 314 315* `deps_info['extra_classpath_jars']: 316For some Java related types, a list of extra `.jar` files to use at build time 317but not at runtime. 318 319## <a name="target_java_binary">Target type `java_binary`</a>: 320 321This type corresponds to a Java binary, which is nothing more than a 322`java_library` target that also provides a main class name. It thus inherits 323all entries from the `java_library` type, and adds: 324 325* `deps_info['main_class']`: 326Name of the main Java class that serves as an entry point for the binary. 327 328* `deps_info['device_classpath']`: 329The classpath used when running a Java or Android binary. Essentially the 330collection of all `deps_info['device_jar_path']` entries for the target and all 331its dependencies. 332 333* `deps_info['all_dex_files']`: 334The list of paths to all `deps_info['dex_path']` entries for all libraries 335that comprise this APK. Valid only for debug builds. 336 337* `deps_info['preferred_dep']`: 338Whether the target should be the preferred dep. This is usually the case when we 339have a java_group that depends on either the public or internal dep accordingly, 340and it is better to depend on the group rather than the underlying dep. Another 341case is for android_library_factory targets, the factory target should be 342preferred instead of the actual implementation. 343 344## <a name="target_robolectric_binary">Target type `robolectric_binary`</a>: 345 346A target type for JUnit-specific binaries. Identical to 347[`java_binary`](#target_java_binary) in the context of `.build_config` files, 348except the name. 349 350 351## <a name="target_java_annotation_processor">Target type \ 352`java_annotation_processor`</a>: 353 354A target type for Java annotation processors. Identical to 355[`java_binary`](#target_java_binary) in the context of `.build_config` files, 356except the name, except that it requires a `deps_info['main_class']` entry. 357 358 359## <a name="target_android_apk">Target type `android_apk`</a>: 360 361Corresponds to an Android APK. Inherits from the 362[`java_binary`](#target_java_binary) type and adds: 363 364* `deps_info['apk_path']`: 365Path to the raw, unsigned, APK generated by this target. 366 367* `deps_info['incremental_apk_path']`: 368Path to the raw, unsigned, incremental APK generated by this target. 369 370* `deps_info['incremental_install_json_path']`: 371Path to the JSON file with per-apk details for incremental install. 372See `build/android/gyp/incremental/write_installer_json.py` for more 373details about its content. 374 375* `deps_info['dist_jar']['all_interface_jars']`: 376For `android_apk` and `dist_jar` targets, a list of all interface jar files 377that will be merged into the final `.jar` file for distribution. 378 379* `deps_info['final_dex']['path']`: 380Path to the final classes.dex file (or classes.zip in case of multi-dex) 381for this APK - only used for proguarded builds. 382 383* `native['libraries']` 384List of native libraries for the primary ABI to be embedded in this APK. 385E.g. [ "libchrome.so" ] (i.e. this doesn't include any ABI sub-directory 386prefix). 387 388* `native['java_libraries_list']` 389The same list as `native['libraries']` as a string holding a Java source 390fragment, e.g. `"{\"chrome\"}"`, without any `lib` prefix, and `.so` 391suffix (as expected by `System.loadLibrary()`). 392 393* `native['second_abi_libraries']` 394List of native libraries for the secondary ABI to be embedded in this APK. 395Empty if only a single ABI is supported. 396 397* `native['loadable_modules']` 398A list of native libraries to store within the APK, in addition to those from 399`native['libraries']`. These correspond to things like the Chromium linker 400or instrumentation libraries. 401 402* `native['secondary_abi_loadable_modules']` 403Secondary ABI version of loadable_modules 404 405* `native['library_always_compress']` 406A list of library files that we always compress. 407 408* `assets` 409A list of assets stored compressed in the APK. Each entry has the format 410`<source-path>:<destination-path>`, where `<source-path>` is relative to 411`$CHROMIUM_OUTPUT_DIR`, and `<destination-path>` is relative to `//assets/` 412within the APK. 413 414NOTE: Not to be confused with the `deps_info['assets']` dictionary that 415belongs to `android_assets` targets only. 416 417* `uncompressed_assets` 418A list of uncompressed assets stored in the APK. Each entry has the format 419`<source-path>:<destination-path>` too. 420 421* `locales_java_list` 422A string holding a Java source fragment that gives the list of locales stored 423uncompressed as android assets. 424 425* `extra_android_manifests` 426A list of `deps_configs['android_manifest]` entries, for all resource 427dependencies for this target. I.e. a list of paths to manifest files for 428all the resources in this APK. These will be merged with the root manifest 429file to generate the final one used to build the APK. 430 431* `java_resources_jars` 432This is a list of `.jar` files whose *Java* resources should be included in 433the final APK. For example, this is used to copy the `.res` files from the 434EMMA Coverage tool. The copy will omit any `.class` file and the top-level 435`//meta-inf/` directory from the input jars. Everything else will be copied 436into the final APK as-is. 437 438NOTE: This has nothing to do with *Android* resources. 439 440* `deps_info['proguard_all_configs']`: 441The collection of all 'deps_info['proguard_configs']` values from this target 442and all its dependencies. 443 444* `deps_info['proguard_classpath_jars']`: 445The collection of all 'deps_info['extra_classpath_jars']` values from all 446dependencies. 447 448* `deps_info['proguard_under_test_mapping']`: 449Applicable to apks with proguard enabled that have an apk_under_test. This is 450the path to the apk_under_test's output proguard .mapping file. 451 452## <a name="target_android_app_bundle_module">Target type \ 453`android_app_bundle_module`</a>: 454 455Corresponds to an Android app bundle module. Very similar to an APK and 456inherits the same fields, except that this does not generate an installable 457file (see `android_app_bundle`), and for the following omitted fields: 458 459* `deps_info['apk_path']`, `deps_info['incremental_apk_path']` and 460 `deps_info['incremental_install_json_path']` are omitted. 461 462* top-level `dist_jar` is omitted as well. 463 464In addition to `android_apk` targets though come these new fields: 465 466* `deps_info['proto_resources_path']`: 467The path of an zip archive containing the APK's resources compiled to the 468protocol buffer format (instead of regular binary xml + resources.arsc). 469 470* `deps_info['module_rtxt_path']`: 471The path of the R.txt file generated when compiling the resources for the bundle 472module. 473 474* `deps_info['module_pathmap_path']`: 475The path of the pathmap file generated when compiling the resources for the 476bundle module, if resource path shortening is enabled. 477 478* `deps_info['base_allowlist_rtxt_path']`: 479Optional path to an R.txt file used as a allowlist for base string resources. 480This means that any string resource listed in this file *and* in 481`deps_info['module_rtxt_path']` will end up in the base split APK of any 482`android_app_bundle` target that uses this target as its base module. 483 484This ensures that such localized strings are available to all bundle installs, 485even when language based splits are enabled (e.g. required for WebView strings 486inside the Monochrome bundle). 487 488 489## <a name="target_android_app_bundle">Target type `android_app_bundle`</a> 490 491This target type corresponds to an Android app bundle, and is built from one 492or more `android_app_bundle_module` targets listed as dependencies. 493 494 495## <a name="target_dist_aar">Target type `dist_aar`</a>: 496 497This type corresponds to a target used to generate an `.aar` archive for 498distribution. The archive's content is determined by the target's dependencies. 499 500This always has the following entries: 501 502 * `deps_info['supports_android']` (always True). 503 * `deps_info['requires_android']` (always True). 504 * `deps_info['proguard_configs']` (optional). 505 506 507## <a name="target_dist_jar">Target type `dist_jar`</a>: 508 509This type is similar to [`dist_aar`](#target_dist_aar) but is not 510Android-specific, and used to create a `.jar` file that can be later 511redistributed. 512 513This always has the following entries: 514 515 * `deps_info['proguard_enabled']` (False by default). 516 * `deps_info['proguard_configs']` (optional). 517 * `deps_info['supports_android']` (True by default). 518 * `deps_info['requires_android']` (False by default). 519 520 521 522## <a name="dict_javac">The `deps_info['javac']` dictionary</a>: 523 524This dictionary appears in Java-related targets (e.g. `java_library`, 525`android_apk` and others), and contains information related to the compilation 526of Java sources, class files, and jars. 527 528* `javac['classpath']` 529The classpath used to compile this target when annotation processors are 530present. 531 532* `javac['interface_classpath']` 533The classpath used to compile this target when annotation processors are 534not present. These are also always used to known when a target needs to be 535rebuilt. 536 537* `javac['processor_classpath']` 538The classpath listing the jars used for annotation processors. I.e. sent as 539`-processorpath` when invoking `javac`. 540 541* `javac['processor_classes']` 542The list of annotation processor main classes. I.e. sent as `-processor' when 543invoking `javac`. 544 545## <a name="android_app_bundle">Target type `android_app_bundle`</a>: 546 547This type corresponds to an Android app bundle (`.aab` file). 548 549--------------- END_MARKDOWN --------------------------------------------------- 550""" 551 552import collections 553import itertools 554import json 555import optparse 556import os 557import shutil 558import sys 559import xml.dom.minidom 560 561from util import build_utils 562from util import resource_utils 563import action_helpers # build_utils adds //build to sys.path. 564 565 566# Types that should never be used as a dependency of another build config. 567_ROOT_TYPES = ('android_apk', 'java_binary', 'java_annotation_processor', 568 'robolectric_binary', 'android_app_bundle') 569# Types that should not allow code deps to pass through. 570_RESOURCE_TYPES = ('android_assets', 'android_resources', 'system_java_library') 571 572# Cache of path -> JSON dict. 573_dep_config_cache = {} 574 575 576class OrderedSet(collections.OrderedDict): 577 @staticmethod 578 def fromkeys(iterable): 579 out = OrderedSet() 580 out.update(iterable) 581 return out 582 583 def add(self, key): 584 self[key] = True 585 586 def update(self, iterable): 587 for v in iterable: 588 self.add(v) 589 590 591def _ExtractMarkdownDocumentation(input_text): 592 """Extract Markdown documentation from a list of input strings lines. 593 594 This generates a list of strings extracted from |input_text|, by looking 595 for '-- BEGIN_MARKDOWN --' and '-- END_MARKDOWN --' line markers.""" 596 in_markdown = False 597 result = [] 598 for line in input_text.splitlines(): 599 if in_markdown: 600 if '-- END_MARKDOWN --' in line: 601 in_markdown = False 602 else: 603 result.append(line) 604 else: 605 if '-- BEGIN_MARKDOWN --' in line: 606 in_markdown = True 607 608 return result 609 610 611class AndroidManifest: 612 def __init__(self, path): 613 self.path = path 614 dom = xml.dom.minidom.parse(path) 615 manifests = dom.getElementsByTagName('manifest') 616 assert len(manifests) == 1 617 self.manifest = manifests[0] 618 619 def GetInstrumentationElements(self): 620 instrumentation_els = self.manifest.getElementsByTagName('instrumentation') 621 if len(instrumentation_els) == 0: 622 return None 623 return instrumentation_els 624 625 def CheckInstrumentationElements(self, expected_package): 626 instrs = self.GetInstrumentationElements() 627 if not instrs: 628 raise Exception('No <instrumentation> elements found in %s' % self.path) 629 for instr in instrs: 630 instrumented_package = instr.getAttributeNS( 631 'http://schemas.android.com/apk/res/android', 'targetPackage') 632 if instrumented_package != expected_package: 633 raise Exception( 634 'Wrong instrumented package. Expected %s, got %s' 635 % (expected_package, instrumented_package)) 636 637 def GetPackageName(self): 638 return self.manifest.getAttribute('package') 639 640 641def GetDepConfigRoot(path): 642 if not path in _dep_config_cache: 643 with open(path) as jsonfile: 644 _dep_config_cache[path] = json.load(jsonfile) 645 return _dep_config_cache[path] 646 647 648def GetDepConfig(path): 649 return GetDepConfigRoot(path)['deps_info'] 650 651 652def DepsOfType(wanted_type, configs): 653 return [c for c in configs if c['type'] == wanted_type] 654 655 656def DepPathsOfType(wanted_type, config_paths): 657 return [p for p in config_paths if GetDepConfig(p)['type'] == wanted_type] 658 659 660def GetAllDepsConfigsInOrder(deps_config_paths, filter_func=None): 661 def apply_filter(paths): 662 if filter_func: 663 return [p for p in paths if filter_func(GetDepConfig(p))] 664 return paths 665 666 def discover(path): 667 config = GetDepConfig(path) 668 all_deps = config['deps_configs'] + config.get('public_deps_configs', []) 669 return apply_filter(all_deps) 670 671 deps_config_paths = apply_filter(deps_config_paths) 672 deps_config_paths = build_utils.GetSortedTransitiveDependencies( 673 deps_config_paths, discover) 674 return deps_config_paths 675 676 677def GetObjectByPath(obj, key_path): 678 """Given an object, return its nth child based on a key path. 679 """ 680 return GetObjectByPath(obj[key_path[0]], key_path[1:]) if key_path else obj 681 682 683def RemoveObjDups(obj, base, *key_path): 684 """Remove array items from an object[*kep_path] that are also 685 contained in the base[*kep_path] (duplicates). 686 """ 687 base_target = set(GetObjectByPath(base, key_path)) 688 target = GetObjectByPath(obj, key_path) 689 target[:] = [x for x in target if x not in base_target] 690 691 692class Deps: 693 def __init__(self, direct_deps_config_paths): 694 self._all_deps_config_paths = GetAllDepsConfigsInOrder( 695 direct_deps_config_paths) 696 self._direct_deps_configs = [ 697 GetDepConfig(p) for p in direct_deps_config_paths 698 ] 699 self._all_deps_configs = [ 700 GetDepConfig(p) for p in self._all_deps_config_paths 701 ] 702 self._direct_deps_config_paths = direct_deps_config_paths 703 704 def All(self, wanted_type=None): 705 if wanted_type is None: 706 return self._all_deps_configs 707 return DepsOfType(wanted_type, self._all_deps_configs) 708 709 def Direct(self, wanted_type=None): 710 if wanted_type is None: 711 return self._direct_deps_configs 712 return DepsOfType(wanted_type, self._direct_deps_configs) 713 714 def AllConfigPaths(self): 715 return self._all_deps_config_paths 716 717 def GradlePrebuiltJarPaths(self): 718 ret = [] 719 720 def helper(cur): 721 for config in cur.Direct('java_library'): 722 if config['is_prebuilt'] or config['gradle_treat_as_prebuilt']: 723 if config['unprocessed_jar_path'] not in ret: 724 ret.append(config['unprocessed_jar_path']) 725 726 helper(self) 727 return ret 728 729 def GradleLibraryProjectDeps(self): 730 ret = [] 731 732 def helper(cur): 733 for config in cur.Direct('java_library'): 734 if config['is_prebuilt']: 735 pass 736 elif config['gradle_treat_as_prebuilt']: 737 all_deps = config['deps_configs'] + config.get( 738 'public_deps_configs', []) 739 helper(Deps(all_deps)) 740 elif config not in ret: 741 ret.append(config) 742 743 helper(self) 744 return ret 745 746 747def _MergeAssets(all_assets): 748 """Merges all assets from the given deps. 749 750 Returns: 751 A tuple of: (compressed, uncompressed, locale_paks) 752 |compressed| and |uncompressed| are lists of "srcPath:zipPath". srcPath is 753 the path of the asset to add, and zipPath is the location within the zip 754 (excluding assets/ prefix). 755 |locale_paks| is a set of all zipPaths that have been marked as 756 treat_as_locale_paks=true. 757 """ 758 compressed = {} 759 uncompressed = {} 760 locale_paks = set() 761 for asset_dep in all_assets: 762 entry = asset_dep['assets'] 763 disable_compression = entry.get('disable_compression') 764 treat_as_locale_paks = entry.get('treat_as_locale_paks') 765 dest_map = uncompressed if disable_compression else compressed 766 other_map = compressed if disable_compression else uncompressed 767 outputs = entry.get('outputs', []) 768 for src, dest in itertools.zip_longest(entry['sources'], outputs): 769 if not dest: 770 dest = os.path.basename(src) 771 # Merge so that each path shows up in only one of the lists, and that 772 # deps of the same target override previous ones. 773 other_map.pop(dest, 0) 774 dest_map[dest] = src 775 if treat_as_locale_paks: 776 locale_paks.add(dest) 777 778 def create_list(asset_map): 779 # Sort to ensure deterministic ordering. 780 items = sorted(asset_map.items()) 781 return [f'{src}:{dest}' for dest, src in items] 782 783 return create_list(compressed), create_list(uncompressed), locale_paks 784 785 786def _ResolveGroupsAndPublicDeps(config_paths): 787 """Returns a list of configs with all groups inlined.""" 788 789 def helper(config_path): 790 config = GetDepConfig(config_path) 791 if config['type'] == 'group': 792 # Groups combine public_deps with deps_configs, so no need to check 793 # public_config_paths separately. 794 return config['deps_configs'] 795 if config['type'] == 'android_resources': 796 # android_resources targets do not support public_deps, but instead treat 797 # all resource deps as public deps. 798 return DepPathsOfType('android_resources', config['deps_configs']) 799 800 return config.get('public_deps_configs', []) 801 802 return build_utils.GetSortedTransitiveDependencies(config_paths, helper) 803 804 805def _DepsFromPaths(dep_paths, 806 target_type, 807 filter_root_targets=True, 808 recursive_resource_deps=False): 809 """Resolves all groups and trims dependency branches that we never want. 810 811 E.g. When a resource or asset depends on an apk target, the intent is to 812 include the .apk as a resource/asset, not to have the apk's classpath added. 813 814 This method is meant to be called to get the top nodes (i.e. closest to 815 current target) that we could then use to get a full transitive dependants 816 list (eg using Deps#all). So filtering single elements out of this list, 817 filters whole branches of dependencies. By resolving groups (i.e. expanding 818 them to their constituents), depending on a group is equivalent to directly 819 depending on each element of that group. 820 """ 821 blocklist = [] 822 allowlist = [] 823 824 # Don't allow root targets to be considered as a dep. 825 if filter_root_targets: 826 blocklist.extend(_ROOT_TYPES) 827 828 # Don't allow java libraries to cross through assets/resources. 829 if target_type in _RESOURCE_TYPES: 830 allowlist.extend(_RESOURCE_TYPES) 831 # Pretend that this target directly depends on all of its transitive 832 # dependencies. 833 if recursive_resource_deps: 834 dep_paths = GetAllDepsConfigsInOrder(dep_paths) 835 # Exclude assets if recursive_resource_deps is set. The 836 # recursive_resource_deps arg is used to pull resources into the base 837 # module to workaround bugs accessing resources in isolated DFMs, but 838 # assets should be kept in the DFMs. 839 blocklist.append('android_assets') 840 841 return _DepsFromPathsWithFilters(dep_paths, blocklist, allowlist) 842 843 844def _FilterConfigPaths(dep_paths, blocklist=None, allowlist=None): 845 if not blocklist and not allowlist: 846 return dep_paths 847 configs = [GetDepConfig(p) for p in dep_paths] 848 if blocklist: 849 configs = [c for c in configs if c['type'] not in blocklist] 850 if allowlist: 851 configs = [c for c in configs if c['type'] in allowlist] 852 853 return [c['path'] for c in configs] 854 855 856def _DepsFromPathsWithFilters(dep_paths, blocklist=None, allowlist=None): 857 """Resolves all groups and trims dependency branches that we never want. 858 859 See _DepsFromPaths. 860 861 |blocklist| if passed, are the types of direct dependencies we do not care 862 about (i.e. tips of branches that we wish to prune). 863 864 |allowlist| if passed, are the only types of direct dependencies we care 865 about (i.e. we wish to prune all other branches that do not start from one of 866 these). 867 """ 868 # Filter both before and after so that public_deps of blocked targets are not 869 # added. 870 allowlist_with_groups = None 871 if allowlist: 872 allowlist_with_groups = set(allowlist) 873 allowlist_with_groups.add('group') 874 dep_paths = _FilterConfigPaths(dep_paths, blocklist, allowlist_with_groups) 875 dep_paths = _ResolveGroupsAndPublicDeps(dep_paths) 876 dep_paths = _FilterConfigPaths(dep_paths, blocklist, allowlist) 877 878 return Deps(dep_paths) 879 880 881def _ExtractSharedLibsFromRuntimeDeps(runtime_deps_file): 882 ret = [] 883 with open(runtime_deps_file) as f: 884 for line in f: 885 line = line.rstrip() 886 if not line.endswith('.so'): 887 continue 888 # Only unstripped .so files are listed in runtime deps. 889 # Convert to the stripped .so by going up one directory. 890 ret.append(os.path.normpath(line.replace('lib.unstripped/', ''))) 891 ret.reverse() 892 return ret 893 894 895def _CreateJavaLibrariesList(library_paths): 896 """Returns a java literal array with the "base" library names: 897 e.g. libfoo.so -> foo 898 """ 899 names = ['"%s"' % os.path.basename(s)[3:-3] for s in library_paths] 900 return ('{%s}' % ','.join(sorted(set(names)))) 901 902 903def _CreateJavaLocaleListFromAssets(assets, locale_paks): 904 """Returns a java literal array from a list of locale assets. 905 906 Args: 907 assets: A list of all APK asset paths in the form 'src:dst' 908 locale_paks: A list of asset paths that correponds to the locale pak 909 files of interest. Each |assets| entry will have its 'dst' part matched 910 against it to determine if they are part of the result. 911 Returns: 912 A string that is a Java source literal array listing the locale names 913 of the corresponding asset files, without directory or .pak suffix. 914 E.g. '{"en-GB", "en-US", "es-ES", "fr", ... }' 915 """ 916 assets_paths = [a.split(':')[1] for a in assets] 917 locales = [os.path.basename(a)[:-4] for a in assets_paths if a in locale_paks] 918 return '{%s}' % ','.join('"%s"' % l for l in sorted(locales)) 919 920 921def _AddJarMapping(jar_to_target, configs): 922 for config in configs: 923 jar = config.get('unprocessed_jar_path') 924 if jar: 925 jar_to_target[jar] = config['gn_target'] 926 for jar in config.get('extra_classpath_jars', []): 927 jar_to_target[jar] = config['gn_target'] 928 929 930def _CompareClasspathPriority(dep): 931 return 1 if dep.get('low_classpath_priority') else 0 932 933 934def _DedupFeatureModuleSharedCode(uses_split_arg, modules, 935 field_names_to_dedup): 936 child_to_ancestors = collections.defaultdict(list) 937 if uses_split_arg: 938 for split_pair in uses_split_arg: 939 child, parent = split_pair.split(':') 940 assert child in modules 941 assert parent in modules 942 child_to_ancestors[child] = [parent] 943 944 # Create a full list of ancestors for each module. 945 for name in modules: 946 if name == 'base': 947 continue 948 curr_name = name 949 while curr_name in child_to_ancestors: 950 parent = child_to_ancestors[curr_name][0] 951 if parent not in child_to_ancestors[name]: 952 child_to_ancestors[name].append(parent) 953 curr_name = parent 954 955 if curr_name != 'base': 956 child_to_ancestors[name].append('base') 957 958 # Strip out duplicates from ancestors. 959 for name, module in modules.items(): 960 if name == 'base': 961 continue 962 # Make sure we get all ancestors, not just direct parent. 963 for ancestor in child_to_ancestors[name]: 964 for f in field_names_to_dedup: 965 if f in module: 966 RemoveObjDups(module, modules[ancestor], f) 967 968 # Strip out duplicates from siblings/cousins. 969 for f in field_names_to_dedup: 970 _PromoteToCommonAncestor(modules, child_to_ancestors, f) 971 972 973def _PromoteToCommonAncestor(modules, child_to_ancestors, field_name): 974 module_to_fields_set = {} 975 for module_name, module in modules.items(): 976 if field_name in module: 977 module_to_fields_set[module_name] = set(module[field_name]) 978 979 seen = set() 980 dupes = set() 981 for fields in module_to_fields_set.values(): 982 new_dupes = seen & fields 983 if new_dupes: 984 dupes |= new_dupes 985 seen |= fields 986 987 for d in dupes: 988 owning_modules = [] 989 for module_name, fields in module_to_fields_set.items(): 990 if d in fields: 991 owning_modules.append(module_name) 992 assert len(owning_modules) >= 2 993 # Rely on the fact that ancestors are inserted from closest to 994 # farthest, where "base" should always be the last element. 995 # Arbitrarily using the first owning module - any would work. 996 for ancestor in child_to_ancestors[owning_modules[0]]: 997 ancestor_is_shared_with_all = True 998 for o in owning_modules[1:]: 999 if ancestor not in child_to_ancestors[o]: 1000 ancestor_is_shared_with_all = False 1001 break 1002 if ancestor_is_shared_with_all: 1003 common_ancestor = ancestor 1004 break 1005 for o in owning_modules: 1006 module_to_fields_set[o].remove(d) 1007 module_to_fields_set[common_ancestor].add(d) 1008 1009 for module_name, module in modules.items(): 1010 if field_name in module: 1011 module[field_name] = sorted(list(module_to_fields_set[module_name])) 1012 1013 1014def _CopyBuildConfigsForDebugging(debug_dir): 1015 shutil.rmtree(debug_dir, ignore_errors=True) 1016 os.makedirs(debug_dir) 1017 for src_path in _dep_config_cache: 1018 dst_path = os.path.join(debug_dir, src_path) 1019 assert dst_path.startswith(debug_dir), dst_path 1020 os.makedirs(os.path.dirname(dst_path), exist_ok=True) 1021 shutil.copy(src_path, dst_path) 1022 print(f'Copied {len(_dep_config_cache)} .build_config.json into {debug_dir}') 1023 1024 1025def main(argv): 1026 parser = optparse.OptionParser() 1027 action_helpers.add_depfile_arg(parser) 1028 parser.add_option('--build-config', help='Path to build_config output.') 1029 parser.add_option('--store-deps-for-debugging-to', 1030 help='Path to copy all transitive build config files to.') 1031 parser.add_option( 1032 '--type', 1033 help='Type of this target (e.g. android_library).') 1034 parser.add_option('--gn-target', help='GN label for this target') 1035 parser.add_option( 1036 '--deps-configs', 1037 help='GN-list of dependent build_config files.') 1038 parser.add_option( 1039 '--annotation-processor-configs', 1040 help='GN-list of build_config files for annotation processors.') 1041 1042 # android_resources options 1043 parser.add_option('--srcjar', help='Path to target\'s resources srcjar.') 1044 parser.add_option('--resources-zip', help='Path to target\'s resources zip.') 1045 parser.add_option('--package-name', 1046 help='Java package name for these resources.') 1047 parser.add_option('--android-manifest', 1048 help='Path to the root android manifest.') 1049 parser.add_option('--merged-android-manifest', 1050 help='Path to the merged android manifest.') 1051 parser.add_option('--resource-dirs', action='append', default=[], 1052 help='GYP-list of resource dirs') 1053 parser.add_option( 1054 '--res-sources-path', 1055 help='Path to file containing a list of paths to resources.') 1056 parser.add_option( 1057 '--resource-overlay', 1058 action='store_true', 1059 help='Whether resources passed in via --resources-zip should override ' 1060 'resources with the same name') 1061 parser.add_option( 1062 '--recursive-resource-deps', 1063 action='store_true', 1064 help='Whether deps should be walked recursively to find resource deps.') 1065 1066 # android_assets options 1067 parser.add_option('--asset-sources', help='List of asset sources.') 1068 parser.add_option('--asset-renaming-sources', 1069 help='List of asset sources with custom destinations.') 1070 parser.add_option('--asset-renaming-destinations', 1071 help='List of asset custom destinations.') 1072 parser.add_option('--disable-asset-compression', action='store_true', 1073 help='Whether to disable asset compression.') 1074 parser.add_option('--treat-as-locale-paks', action='store_true', 1075 help='Consider the assets as locale paks in BuildConfig.java') 1076 1077 # java library and group options 1078 parser.add_option('--preferred-dep', 1079 action='store_true', 1080 help='Whether the target should be preferred as a dep.') 1081 1082 # java library options 1083 parser.add_option('--public-deps-configs', 1084 help='GN list of config files of deps which are exposed as ' 1085 'part of the target\'s public API.') 1086 parser.add_option('--aar-path', help='Path to containing .aar file.') 1087 parser.add_option('--device-jar-path', help='Path to .jar for dexing.') 1088 parser.add_option('--host-jar-path', help='Path to .jar for java_binary.') 1089 parser.add_option('--unprocessed-jar-path', 1090 help='Path to the .jar to use for javac classpath purposes.') 1091 parser.add_option( 1092 '--interface-jar-path', 1093 help='Path to the interface .jar to use for javac classpath purposes.') 1094 parser.add_option('--is-prebuilt', action='store_true', 1095 help='Whether the jar was compiled or pre-compiled.') 1096 parser.add_option('--target-sources-file', help='Path to .sources file') 1097 parser.add_option('--bundled-srcjars', 1098 help='GYP-list of .srcjars that have been included in this java_library.') 1099 parser.add_option('--supports-android', action='store_true', 1100 help='Whether this library supports running on the Android platform.') 1101 parser.add_option('--requires-android', action='store_true', 1102 help='Whether this library requires running on the Android platform.') 1103 parser.add_option('--bypass-platform-checks', action='store_true', 1104 help='Bypass checks for support/require Android platform.') 1105 parser.add_option('--extra-classpath-jars', 1106 help='GYP-list of .jar files to include on the classpath when compiling, ' 1107 'but not to include in the final binary.') 1108 parser.add_option( 1109 '--low-classpath-priority', 1110 action='store_true', 1111 help='Indicates that the library should be placed at the end of the ' 1112 'classpath.') 1113 parser.add_option( 1114 '--mergeable-android-manifests', 1115 help='GN-list of AndroidManifest.xml to include in manifest merging.') 1116 parser.add_option('--gradle-treat-as-prebuilt', action='store_true', 1117 help='Whether this library should be treated as a prebuilt library by ' 1118 'generate_gradle.py.') 1119 parser.add_option('--main-class', 1120 help='Main class for java_binary or java_annotation_processor targets.') 1121 parser.add_option('--java-resources-jar-path', 1122 help='Path to JAR that contains java resources. Everything ' 1123 'from this JAR except meta-inf/ content and .class files ' 1124 'will be added to the final APK.') 1125 parser.add_option( 1126 '--non-chromium-code', 1127 action='store_true', 1128 help='True if a java library is not chromium code, used for lint.') 1129 1130 # robolectric_library options 1131 parser.add_option('--is-robolectric', 1132 action='store_true', 1133 help='Whether this is a host side android test library.') 1134 1135 # android library options 1136 parser.add_option('--dex-path', help='Path to target\'s dex output.') 1137 1138 # native library options 1139 parser.add_option('--shared-libraries-runtime-deps', 1140 help='Path to file containing runtime deps for shared ' 1141 'libraries.') 1142 parser.add_option( 1143 '--loadable-modules', 1144 action='append', 1145 help='GN-list of native libraries for primary ' 1146 'android-abi. Can be specified multiple times.', 1147 default=[]) 1148 parser.add_option('--secondary-abi-shared-libraries-runtime-deps', 1149 help='Path to file containing runtime deps for secondary ' 1150 'abi shared libraries.') 1151 parser.add_option( 1152 '--secondary-abi-loadable-modules', 1153 action='append', 1154 help='GN-list of native libraries for secondary ' 1155 'android-abi. Can be specified multiple times.', 1156 default=[]) 1157 parser.add_option( 1158 '--native-lib-placeholders', 1159 action='append', 1160 help='GN-list of native library placeholders to add.', 1161 default=[]) 1162 parser.add_option( 1163 '--secondary-native-lib-placeholders', 1164 action='append', 1165 help='GN-list of native library placeholders to add ' 1166 'for the secondary android-abi.', 1167 default=[]) 1168 parser.add_option('--uncompress-shared-libraries', default=False, 1169 action='store_true', 1170 help='Whether to store native libraries uncompressed') 1171 parser.add_option( 1172 '--library-always-compress', 1173 help='The list of library files that we always compress.') 1174 1175 # apk options 1176 parser.add_option('--apk-path', help='Path to the target\'s apk output.') 1177 parser.add_option('--incremental-apk-path', 1178 help="Path to the target's incremental apk output.") 1179 parser.add_option('--incremental-install-json-path', 1180 help="Path to the target's generated incremental install " 1181 "json.") 1182 parser.add_option( 1183 '--tested-apk-config', 1184 help='Path to the build config of the tested apk (for an instrumentation ' 1185 'test apk).') 1186 parser.add_option( 1187 '--proguard-enabled', 1188 action='store_true', 1189 help='Whether proguard is enabled for this apk or bundle module.') 1190 parser.add_option( 1191 '--proguard-configs', 1192 help='GN-list of proguard flag files to use in final apk.') 1193 parser.add_option( 1194 '--proguard-mapping-path', help='Path to jar created by ProGuard step') 1195 1196 # apk options that are static library specific 1197 parser.add_option( 1198 '--static-library-dependent-configs', 1199 help='GN list of .build_configs of targets that use this target as a ' 1200 'static library.') 1201 1202 # options shared between android_resources and apk targets 1203 parser.add_option('--r-text-path', help='Path to target\'s R.txt file.') 1204 1205 parser.add_option('--fail', 1206 help='GN-list of error message lines to fail with.') 1207 1208 parser.add_option('--final-dex-path', 1209 help='Path to final input classes.dex (or classes.zip) to ' 1210 'use in final apk.') 1211 parser.add_option('--res-size-info', help='Path to .ap_.info') 1212 parser.add_option('--apk-proto-resources', 1213 help='Path to resources compiled in protocol buffer format ' 1214 ' for this apk.') 1215 parser.add_option( 1216 '--module-pathmap-path', 1217 help='Path to pathmap file for resource paths in a bundle module.') 1218 parser.add_option( 1219 '--base-allowlist-rtxt-path', 1220 help='Path to R.txt file for the base resources allowlist.') 1221 1222 parser.add_option('--generate-markdown-format-doc', action='store_true', 1223 help='Dump the Markdown .build_config format documentation ' 1224 'then exit immediately.') 1225 1226 parser.add_option('--module-name', help='The name of this feature module.') 1227 parser.add_option( 1228 '--base-module-build-config', 1229 help='Path to the base module\'s build config ' 1230 'if this is a feature module.') 1231 parser.add_option('--parent-module-build-config', 1232 help='Path to the parent module\'s build config ' 1233 'when not using base module as parent.') 1234 1235 parser.add_option( 1236 '--module-build-configs', 1237 help='For bundles, the paths of all non-async module .build_configs ' 1238 'for modules that are part of the bundle.') 1239 parser.add_option( 1240 '--uses-split', 1241 action='append', 1242 help='List of name pairs separated by : mapping a feature module to a ' 1243 'dependent feature module.') 1244 1245 parser.add_option( 1246 '--trace-events-jar-dir', 1247 help='Directory of rewritten .jar files for trace event rewriting.') 1248 1249 parser.add_option('--version-name', help='Version name for this APK.') 1250 parser.add_option('--version-code', help='Version code for this APK.') 1251 1252 options, args = parser.parse_args(argv) 1253 1254 if args: 1255 parser.error('No positional arguments should be given.') 1256 1257 if options.generate_markdown_format_doc: 1258 doc_lines = _ExtractMarkdownDocumentation(__doc__) 1259 for line in doc_lines: 1260 print(line) 1261 return 0 1262 1263 if options.fail: 1264 parser.error('\n'.join(action_helpers.parse_gn_list(options.fail))) 1265 1266 lib_options = ['unprocessed_jar_path', 'interface_jar_path'] 1267 device_lib_options = ['device_jar_path', 'dex_path'] 1268 required_options_map = { 1269 'android_apk': ['build_config'] + lib_options + device_lib_options, 1270 'android_app_bundle_module': 1271 ['build_config', 'res_size_info'] + lib_options + device_lib_options, 1272 'android_assets': ['build_config'], 1273 'android_resources': ['build_config', 'resources_zip'], 1274 'dist_aar': ['build_config'], 1275 'dist_jar': ['build_config'], 1276 'group': ['build_config'], 1277 'java_annotation_processor': ['build_config', 'main_class'], 1278 'java_binary': ['build_config'], 1279 'java_library': ['build_config', 'host_jar_path'] + lib_options, 1280 'robolectric_binary': ['build_config'], 1281 'system_java_library': ['build_config', 'unprocessed_jar_path'], 1282 'android_app_bundle': ['build_config', 'module_build_configs'], 1283 } 1284 required_options = required_options_map.get(options.type) 1285 if not required_options: 1286 raise Exception('Unknown type: <%s>' % options.type) 1287 1288 build_utils.CheckOptions(options, parser, required_options) 1289 1290 if options.type != 'android_app_bundle_module': 1291 if options.apk_proto_resources: 1292 raise Exception('--apk-proto-resources can only be used with ' 1293 '--type=android_app_bundle_module') 1294 if options.module_pathmap_path: 1295 raise Exception('--module-pathmap-path can only be used with ' 1296 '--type=android_app_bundle_module') 1297 if options.base_allowlist_rtxt_path: 1298 raise Exception('--base-allowlist-rtxt-path can only be used with ' 1299 '--type=android_app_bundle_module') 1300 if options.module_name: 1301 raise Exception('--module-name can only be used with ' 1302 '--type=android_app_bundle_module') 1303 1304 is_apk_or_module_target = options.type in ('android_apk', 1305 'android_app_bundle_module') 1306 1307 if not is_apk_or_module_target: 1308 if options.library_always_compress: 1309 raise Exception( 1310 '--library-always-compress can only be used with --type=android_apk ' 1311 'or --type=android_app_bundle_module') 1312 1313 if options.device_jar_path and not options.dex_path: 1314 raise Exception('java_library that supports Android requires a dex path.') 1315 if any(getattr(options, x) for x in lib_options): 1316 for attr in lib_options: 1317 if not getattr(options, attr): 1318 raise('Expected %s to be set.' % attr) 1319 1320 if options.requires_android and not options.supports_android: 1321 raise Exception( 1322 '--supports-android is required when using --requires-android') 1323 1324 is_java_target = options.type in ('java_binary', 'robolectric_binary', 1325 'java_annotation_processor', 'java_library', 1326 'android_apk', 'dist_aar', 'dist_jar', 1327 'system_java_library', 1328 'android_app_bundle_module') 1329 1330 deps_configs_paths = action_helpers.parse_gn_list(options.deps_configs) 1331 public_deps_configs_paths = action_helpers.parse_gn_list( 1332 options.public_deps_configs) 1333 deps_configs_paths += public_deps_configs_paths 1334 deps = _DepsFromPaths(deps_configs_paths, 1335 options.type, 1336 recursive_resource_deps=options.recursive_resource_deps) 1337 public_deps = _DepsFromPaths(public_deps_configs_paths, options.type) 1338 processor_deps = _DepsFromPaths(action_helpers.parse_gn_list( 1339 options.annotation_processor_configs or ''), 1340 options.type, 1341 filter_root_targets=False) 1342 1343 all_inputs = (deps.AllConfigPaths() + processor_deps.AllConfigPaths()) 1344 1345 if options.recursive_resource_deps: 1346 # Include java_library targets since changes to these targets can remove 1347 # resource deps from the build, which would require rebuilding this target's 1348 # build config file: crbug.com/1168655. 1349 recursive_java_deps = _DepsFromPathsWithFilters( 1350 GetAllDepsConfigsInOrder(deps_configs_paths), 1351 allowlist=['java_library']) 1352 all_inputs.extend(recursive_java_deps.AllConfigPaths()) 1353 1354 system_library_deps = deps.Direct('system_java_library') 1355 all_deps = deps.All() 1356 all_library_deps = deps.All('java_library') 1357 1358 if options.type == 'java_library': 1359 # For Java libraries, restrict to resource targets that are direct deps, or 1360 # are indirect via other resource targets. 1361 # The indirect-through-other-targets ones are picked up because 1362 # _ResolveGroupsAndPublicDeps() treats resource deps of resource targets as 1363 # public_deps. 1364 all_resources_deps = deps.Direct('android_resources') 1365 else: 1366 all_resources_deps = deps.All('android_resources') 1367 1368 if options.type == 'android_resources' and options.recursive_resource_deps: 1369 # android_resources targets that want recursive resource deps also need to 1370 # collect package_names from all library deps. This ensures the R.java files 1371 # for these libraries will get pulled in along with the resources. 1372 android_resources_library_deps = _DepsFromPathsWithFilters( 1373 deps_configs_paths, allowlist=['java_library']).All('java_library') 1374 1375 base_module_build_config = None 1376 if options.base_module_build_config: 1377 base_module_build_config = GetDepConfigRoot( 1378 options.base_module_build_config) 1379 parent_module_build_config = base_module_build_config 1380 if options.parent_module_build_config: 1381 parent_module_build_config = GetDepConfigRoot( 1382 options.parent_module_build_config) 1383 1384 # Initialize some common config. 1385 # Any value that needs to be queryable by dependents must go within deps_info. 1386 config = { 1387 'deps_info': { 1388 'name': os.path.basename(options.build_config), 1389 'path': options.build_config, 1390 'type': options.type, 1391 'gn_target': options.gn_target, 1392 'chromium_code': not options.non_chromium_code, 1393 }, 1394 # Info needed only by generate_gradle.py. 1395 'gradle': {} 1396 } 1397 deps_info = config['deps_info'] 1398 gradle = config['gradle'] 1399 1400 # The paths we record as deps can differ from deps_config_paths: 1401 # 1) Paths can be removed when blocked by _ROOT_TYPES / _RESOURCE_TYPES. 1402 # 2) Paths can be added when promoted from group deps or public_deps of deps. 1403 # Deps are promoted from groups/public_deps in order to make the filtering 1404 # of 1) work through group() targets (which themselves are not resource 1405 # targets, but should be treated as such when depended on by a resource 1406 # target. A more involved filtering implementation could work to maintain 1407 # the semantics of 1) without the need to promote deps, but we've avoided 1408 # such an undertaking so far. 1409 public_deps_set = set() 1410 if public_deps_configs_paths: 1411 deps_info['public_deps_configs'] = [d['path'] for d in public_deps.Direct()] 1412 public_deps_set = set(deps_info['public_deps_configs']) 1413 1414 deps_info['deps_configs'] = [ 1415 d['path'] for d in deps.Direct() if d['path'] not in public_deps_set 1416 ] 1417 1418 if options.type == 'android_apk' and options.tested_apk_config: 1419 tested_apk_deps = Deps([options.tested_apk_config]) 1420 tested_apk_config = tested_apk_deps.Direct()[0] 1421 gradle['apk_under_test'] = tested_apk_config['name'] 1422 1423 if options.type == 'android_app_bundle_module': 1424 deps_info['module_name'] = options.module_name 1425 1426 # Required for generating gradle files. 1427 if options.type == 'java_library': 1428 deps_info['is_prebuilt'] = bool(options.is_prebuilt) 1429 deps_info['gradle_treat_as_prebuilt'] = options.gradle_treat_as_prebuilt 1430 1431 if options.preferred_dep: 1432 deps_info['preferred_dep'] = bool(options.preferred_dep) 1433 1434 if options.android_manifest: 1435 deps_info['android_manifest'] = options.android_manifest 1436 1437 if options.merged_android_manifest: 1438 deps_info['merged_android_manifest'] = options.merged_android_manifest 1439 1440 if options.bundled_srcjars: 1441 deps_info['bundled_srcjars'] = action_helpers.parse_gn_list( 1442 options.bundled_srcjars) 1443 1444 if options.target_sources_file: 1445 deps_info['target_sources_file'] = options.target_sources_file 1446 1447 if is_java_target: 1448 if options.main_class: 1449 deps_info['main_class'] = options.main_class 1450 1451 dependent_prebuilt_jars = deps.GradlePrebuiltJarPaths() 1452 dependent_prebuilt_jars.sort() 1453 if dependent_prebuilt_jars: 1454 gradle['dependent_prebuilt_jars'] = dependent_prebuilt_jars 1455 1456 dependent_android_projects = [] 1457 dependent_java_projects = [] 1458 for c in deps.GradleLibraryProjectDeps(): 1459 if c['requires_android']: 1460 dependent_android_projects.append(c['path']) 1461 else: 1462 dependent_java_projects.append(c['path']) 1463 1464 gradle['dependent_android_projects'] = dependent_android_projects 1465 gradle['dependent_java_projects'] = dependent_java_projects 1466 1467 if options.r_text_path: 1468 deps_info['r_text_path'] = options.r_text_path 1469 1470 if is_apk_or_module_target or options.type in ('group', 'java_library', 1471 'robolectric_binary'): 1472 if options.apk_proto_resources: 1473 deps_info['proto_resources_path'] = options.apk_proto_resources 1474 1475 deps_info['version_name'] = options.version_name 1476 deps_info['version_code'] = options.version_code 1477 if options.module_pathmap_path: 1478 deps_info['module_pathmap_path'] = options.module_pathmap_path 1479 else: 1480 # Ensure there is an entry, even if it is empty, for modules 1481 # that have not enabled resource path shortening. Otherwise 1482 # build_utils.ExpandFileArgs fails. 1483 deps_info['module_pathmap_path'] = '' 1484 1485 if options.base_allowlist_rtxt_path: 1486 deps_info['base_allowlist_rtxt_path'] = options.base_allowlist_rtxt_path 1487 else: 1488 # Ensure there is an entry, even if it is empty, for modules 1489 # that don't need such a allowlist. 1490 deps_info['base_allowlist_rtxt_path'] = '' 1491 1492 if is_java_target: 1493 deps_info['requires_android'] = bool(options.requires_android) 1494 deps_info['supports_android'] = bool(options.supports_android) 1495 1496 # robolectric is special in that its an android target that runs on host. 1497 # You are allowed to depend on both android |deps_require_android| and 1498 # non-android |deps_not_support_android| targets. 1499 if not options.bypass_platform_checks and not options.is_robolectric: 1500 deps_require_android = all_resources_deps + [ 1501 d for d in all_library_deps if d['requires_android'] 1502 ] 1503 deps_not_support_android = [ 1504 d for d in all_library_deps if not d['supports_android'] 1505 ] 1506 1507 if deps_require_android and not options.requires_android: 1508 raise Exception( 1509 'Some deps require building for the Android platform:\n' + 1510 '\n'.join('* ' + d['gn_target'] for d in deps_require_android)) 1511 1512 if deps_not_support_android and options.supports_android: 1513 raise Exception('Not all deps support the Android platform:\n' + 1514 '\n'.join('* ' + d['gn_target'] 1515 for d in deps_not_support_android)) 1516 1517 if is_apk_or_module_target or options.type == 'dist_jar': 1518 all_dex_files = [c['dex_path'] for c in all_library_deps] 1519 1520 if is_java_target: 1521 # Classpath values filled in below (after applying tested_apk_config). 1522 config['javac'] = {} 1523 if options.aar_path: 1524 deps_info['aar_path'] = options.aar_path 1525 if options.unprocessed_jar_path: 1526 deps_info['unprocessed_jar_path'] = options.unprocessed_jar_path 1527 deps_info['interface_jar_path'] = options.interface_jar_path 1528 if options.device_jar_path: 1529 deps_info['device_jar_path'] = options.device_jar_path 1530 if options.host_jar_path: 1531 deps_info['host_jar_path'] = options.host_jar_path 1532 if options.dex_path: 1533 deps_info['dex_path'] = options.dex_path 1534 if is_apk_or_module_target: 1535 all_dex_files.append(options.dex_path) 1536 if options.low_classpath_priority: 1537 deps_info['low_classpath_priority'] = True 1538 if options.type == 'android_apk': 1539 deps_info['apk_path'] = options.apk_path 1540 deps_info['incremental_apk_path'] = options.incremental_apk_path 1541 deps_info['incremental_install_json_path'] = ( 1542 options.incremental_install_json_path) 1543 1544 if options.type == 'android_assets': 1545 all_asset_sources = [] 1546 if options.asset_renaming_sources: 1547 all_asset_sources.extend( 1548 action_helpers.parse_gn_list(options.asset_renaming_sources)) 1549 if options.asset_sources: 1550 all_asset_sources.extend( 1551 action_helpers.parse_gn_list(options.asset_sources)) 1552 1553 deps_info['assets'] = { 1554 'sources': all_asset_sources 1555 } 1556 if options.asset_renaming_destinations: 1557 deps_info['assets']['outputs'] = (action_helpers.parse_gn_list( 1558 options.asset_renaming_destinations)) 1559 if options.disable_asset_compression: 1560 deps_info['assets']['disable_compression'] = True 1561 if options.treat_as_locale_paks: 1562 deps_info['assets']['treat_as_locale_paks'] = True 1563 1564 if options.type == 'android_resources': 1565 deps_info['resources_zip'] = options.resources_zip 1566 if options.resource_overlay: 1567 deps_info['resource_overlay'] = True 1568 if options.srcjar: 1569 deps_info['srcjar'] = options.srcjar 1570 if options.android_manifest: 1571 manifest = AndroidManifest(options.android_manifest) 1572 deps_info['package_name'] = manifest.GetPackageName() 1573 if options.package_name: 1574 deps_info['package_name'] = options.package_name 1575 deps_info['res_sources_path'] = '' 1576 if options.res_sources_path: 1577 deps_info['res_sources_path'] = options.res_sources_path 1578 1579 if (options.requires_android 1580 and options.type == 'java_library') or options.is_robolectric: 1581 if options.package_name: 1582 deps_info['package_name'] = options.package_name 1583 1584 if options.type in ('android_resources', 'android_apk', 'robolectric_binary', 1585 'dist_aar', 'android_app_bundle_module', 'java_library'): 1586 dependency_zips = [] 1587 dependency_zip_overlays = [] 1588 for c in all_resources_deps: 1589 if not c['resources_zip']: 1590 continue 1591 1592 dependency_zips.append(c['resources_zip']) 1593 if c.get('resource_overlay'): 1594 dependency_zip_overlays.append(c['resources_zip']) 1595 1596 extra_package_names = [] 1597 1598 if options.type != 'android_resources': 1599 extra_package_names = [ 1600 c['package_name'] for c in all_resources_deps if 'package_name' in c 1601 ] 1602 if options.package_name: 1603 extra_package_names += [options.package_name] 1604 1605 # android_resources targets which specified recursive_resource_deps may 1606 # have extra_package_names. 1607 for resources_dep in all_resources_deps: 1608 extra_package_names.extend(resources_dep['extra_package_names']) 1609 1610 # In final types (i.e. apks and modules) that create real R.java files, 1611 # they must collect package names from java_libraries as well. 1612 # https://crbug.com/1073476 1613 if options.type != 'java_library': 1614 extra_package_names.extend([ 1615 c['package_name'] for c in all_library_deps if 'package_name' in c 1616 ]) 1617 elif options.recursive_resource_deps: 1618 # Pull extra_package_names from library deps if recursive resource deps 1619 # are required. 1620 extra_package_names = [ 1621 c['package_name'] for c in android_resources_library_deps 1622 if 'package_name' in c 1623 ] 1624 config['deps_info']['includes_recursive_resources'] = True 1625 1626 if options.type in ('dist_aar', 'java_library'): 1627 r_text_files = [ 1628 c['r_text_path'] for c in all_resources_deps if 'r_text_path' in c 1629 ] 1630 deps_info['dependency_r_txt_files'] = r_text_files 1631 1632 if options.type == 'android_apk' and options.tested_apk_config: 1633 config['deps_info']['arsc_package_name'] = ( 1634 tested_apk_config['package_name']) 1635 # We should not shadow the actual R.java files of the apk_under_test by 1636 # creating new R.java files with the same package names in the tested apk. 1637 extra_package_names = [ 1638 package for package in extra_package_names 1639 if package not in tested_apk_config['extra_package_names'] 1640 ] 1641 if options.res_size_info: 1642 config['deps_info']['res_size_info'] = options.res_size_info 1643 1644 # Safe to sort: Build checks that non-overlay resource have no overlap. 1645 dependency_zips.sort() 1646 config['deps_info']['dependency_zips'] = dependency_zips 1647 config['deps_info']['dependency_zip_overlays'] = dependency_zip_overlays 1648 # Order doesn't matter, so make stable. 1649 extra_package_names.sort() 1650 config['deps_info']['extra_package_names'] = extra_package_names 1651 1652 # These are .jars to add to javac classpath but not to runtime classpath. 1653 extra_classpath_jars = action_helpers.parse_gn_list( 1654 options.extra_classpath_jars) 1655 if extra_classpath_jars: 1656 extra_classpath_jars.sort() 1657 deps_info['extra_classpath_jars'] = extra_classpath_jars 1658 1659 mergeable_android_manifests = action_helpers.parse_gn_list( 1660 options.mergeable_android_manifests) 1661 mergeable_android_manifests.sort() 1662 if mergeable_android_manifests: 1663 deps_info['mergeable_android_manifests'] = mergeable_android_manifests 1664 1665 extra_proguard_classpath_jars = [] 1666 proguard_configs = action_helpers.parse_gn_list(options.proguard_configs) 1667 if proguard_configs: 1668 # Make a copy of |proguard_configs| since it's mutated below. 1669 deps_info['proguard_configs'] = list(proguard_configs) 1670 1671 1672 if is_java_target: 1673 classpath_direct_deps = deps.Direct() 1674 classpath_direct_library_deps = deps.Direct('java_library') 1675 1676 # The classpath used to compile this target when annotation processors are 1677 # present. 1678 javac_classpath = set(c['unprocessed_jar_path'] 1679 for c in classpath_direct_library_deps) 1680 # The classpath used to compile this target when annotation processors are 1681 # not present. These are also always used to know when a target needs to be 1682 # rebuilt. 1683 javac_interface_classpath = set(c['interface_jar_path'] 1684 for c in classpath_direct_library_deps) 1685 1686 # Preserve order of |all_library_deps|. Move low priority libraries to the 1687 # end of the classpath. 1688 all_library_deps_sorted_for_classpath = sorted( 1689 all_library_deps[::-1], key=_CompareClasspathPriority) 1690 1691 # The classpath used for bytecode-rewritting. 1692 javac_full_classpath = OrderedSet.fromkeys( 1693 c['unprocessed_jar_path'] 1694 for c in all_library_deps_sorted_for_classpath) 1695 # The classpath used for error prone. 1696 javac_full_interface_classpath = OrderedSet.fromkeys( 1697 c['interface_jar_path'] for c in all_library_deps_sorted_for_classpath) 1698 1699 # Adding base module to classpath to compile against its R.java file 1700 if base_module_build_config: 1701 javac_full_classpath.add( 1702 base_module_build_config['deps_info']['unprocessed_jar_path']) 1703 javac_full_interface_classpath.add( 1704 base_module_build_config['deps_info']['interface_jar_path']) 1705 # Turbine now compiles headers against only the direct classpath, so the 1706 # base module's interface jar must be on the direct interface classpath. 1707 javac_interface_classpath.add( 1708 base_module_build_config['deps_info']['interface_jar_path']) 1709 1710 for dep in classpath_direct_deps: 1711 if 'extra_classpath_jars' in dep: 1712 javac_classpath.update(dep['extra_classpath_jars']) 1713 javac_interface_classpath.update(dep['extra_classpath_jars']) 1714 for dep in all_deps: 1715 if 'extra_classpath_jars' in dep: 1716 javac_full_classpath.update(dep['extra_classpath_jars']) 1717 javac_full_interface_classpath.update(dep['extra_classpath_jars']) 1718 1719 # TODO(agrieve): Might be less confusing to fold these into bootclasspath. 1720 # Deps to add to the compile-time classpath (but not the runtime classpath). 1721 # These are jars specified by input_jars_paths that almost never change. 1722 # Just add them directly to all the classpaths. 1723 if options.extra_classpath_jars: 1724 javac_classpath.update(extra_classpath_jars) 1725 javac_interface_classpath.update(extra_classpath_jars) 1726 javac_full_classpath.update(extra_classpath_jars) 1727 javac_full_interface_classpath.update(extra_classpath_jars) 1728 1729 if is_java_target or options.type == 'android_app_bundle': 1730 # The classpath to use to run this target (or as an input to ProGuard). 1731 device_classpath = [] 1732 if is_java_target and options.device_jar_path: 1733 device_classpath.append(options.device_jar_path) 1734 device_classpath.extend( 1735 c.get('device_jar_path') for c in all_library_deps 1736 if c.get('device_jar_path')) 1737 if options.type == 'android_app_bundle': 1738 for d in deps.Direct('android_app_bundle_module'): 1739 device_classpath.extend(c for c in d.get('device_classpath', []) 1740 if c not in device_classpath) 1741 1742 if options.type in ('dist_jar', 'java_binary', 'robolectric_binary'): 1743 # The classpath to use to run this target. 1744 host_classpath = [] 1745 if options.host_jar_path: 1746 host_classpath.append(options.host_jar_path) 1747 host_classpath.extend(c['host_jar_path'] for c in all_library_deps) 1748 deps_info['host_classpath'] = host_classpath 1749 1750 # We allow lint to be run on android_apk targets, so we collect lint 1751 # artifacts for them. 1752 # We allow lint to be run on android_app_bundle targets, so we need to 1753 # collect lint artifacts for the android_app_bundle_module targets that the 1754 # bundle includes. Different android_app_bundle targets may include different 1755 # android_app_bundle_module targets, so the bundle needs to be able to 1756 # de-duplicate these lint artifacts. 1757 if options.type in ('android_app_bundle_module', 'android_apk'): 1758 # Collect all sources and resources at the apk/bundle_module level. 1759 lint_aars = set() 1760 lint_srcjars = set() 1761 lint_sources = set() 1762 lint_resource_sources = set() 1763 lint_resource_zips = set() 1764 1765 if options.target_sources_file: 1766 lint_sources.add(options.target_sources_file) 1767 if options.bundled_srcjars: 1768 lint_srcjars.update(deps_info['bundled_srcjars']) 1769 for c in all_library_deps: 1770 if c['chromium_code'] and c['requires_android']: 1771 if 'target_sources_file' in c: 1772 lint_sources.add(c['target_sources_file']) 1773 lint_srcjars.update(c['bundled_srcjars']) 1774 if 'aar_path' in c: 1775 lint_aars.add(c['aar_path']) 1776 1777 if options.res_sources_path: 1778 lint_resource_sources.add(options.res_sources_path) 1779 if options.resources_zip: 1780 lint_resource_zips.add(options.resources_zip) 1781 for c in all_resources_deps: 1782 if c['chromium_code']: 1783 # Prefer res_sources_path to resources_zips so that lint errors have 1784 # real paths and to avoid needing to extract during lint. 1785 if c['res_sources_path']: 1786 lint_resource_sources.add(c['res_sources_path']) 1787 else: 1788 lint_resource_zips.add(c['resources_zip']) 1789 1790 deps_info['lint_aars'] = sorted(lint_aars) 1791 deps_info['lint_srcjars'] = sorted(lint_srcjars) 1792 deps_info['lint_sources'] = sorted(lint_sources) 1793 deps_info['lint_resource_sources'] = sorted(lint_resource_sources) 1794 deps_info['lint_resource_zips'] = sorted(lint_resource_zips) 1795 deps_info['lint_extra_android_manifests'] = [] 1796 1797 if options.type == 'android_apk': 1798 assert options.android_manifest, 'Android APKs must define a manifest' 1799 deps_info['lint_android_manifest'] = options.android_manifest 1800 1801 if options.type == 'android_app_bundle': 1802 module_config_paths = action_helpers.parse_gn_list( 1803 options.module_build_configs) 1804 module_configs = [GetDepConfig(c) for c in module_config_paths] 1805 module_configs_by_name = {d['module_name']: d for d in module_configs} 1806 per_module_fields = [ 1807 'device_classpath', 'trace_event_rewritten_device_classpath', 1808 'all_dex_files' 1809 ] 1810 lint_aars = set() 1811 lint_srcjars = set() 1812 lint_sources = set() 1813 lint_resource_sources = set() 1814 lint_resource_zips = set() 1815 lint_extra_android_manifests = set() 1816 config['modules'] = {} 1817 modules = config['modules'] 1818 for n, c in module_configs_by_name.items(): 1819 if n == 'base': 1820 assert 'base_module_config' not in deps_info, ( 1821 'Must have exactly 1 base module!') 1822 deps_info['package_name'] = c['package_name'] 1823 deps_info['version_code'] = c['version_code'] 1824 deps_info['version_name'] = c['version_name'] 1825 deps_info['base_module_config'] = c['path'] 1826 # Use the base module's android manifest for linting. 1827 deps_info['lint_android_manifest'] = c['android_manifest'] 1828 else: 1829 lint_extra_android_manifests.add(c['android_manifest']) 1830 lint_aars.update(c['lint_aars']) 1831 lint_srcjars.update(c['lint_srcjars']) 1832 lint_sources.update(c['lint_sources']) 1833 lint_resource_sources.update(c['lint_resource_sources']) 1834 lint_resource_zips.update(c['lint_resource_zips']) 1835 module = modules[n] = {} 1836 for f in per_module_fields: 1837 if f in c: 1838 module[f] = c[f] 1839 deps_info['lint_aars'] = sorted(lint_aars) 1840 deps_info['lint_srcjars'] = sorted(lint_srcjars) 1841 deps_info['lint_sources'] = sorted(lint_sources) 1842 deps_info['lint_resource_sources'] = sorted(lint_resource_sources) 1843 deps_info['lint_resource_zips'] = sorted(lint_resource_zips) 1844 deps_info['lint_extra_android_manifests'] = sorted( 1845 lint_extra_android_manifests) 1846 1847 _DedupFeatureModuleSharedCode(options.uses_split, modules, 1848 per_module_fields) 1849 1850 system_jars = [c['unprocessed_jar_path'] for c in system_library_deps] 1851 system_interface_jars = [c['interface_jar_path'] for c in system_library_deps] 1852 if system_library_deps: 1853 config['android'] = {} 1854 config['android']['sdk_interface_jars'] = system_interface_jars 1855 config['android']['sdk_jars'] = system_jars 1856 1857 if options.type in ('android_apk', 'dist_aar', 1858 'dist_jar', 'android_app_bundle_module', 'android_app_bundle'): 1859 for c in all_deps: 1860 proguard_configs.extend(c.get('proguard_configs', [])) 1861 extra_proguard_classpath_jars.extend(c.get('extra_classpath_jars', [])) 1862 if options.type == 'android_app_bundle': 1863 for c in deps.Direct('android_app_bundle_module'): 1864 proguard_configs.extend(p for p in c.get('proguard_configs', [])) 1865 if options.type == 'android_app_bundle': 1866 for d in deps.Direct('android_app_bundle_module'): 1867 extra_proguard_classpath_jars.extend( 1868 c for c in d.get('proguard_classpath_jars', []) 1869 if c not in extra_proguard_classpath_jars) 1870 1871 if options.type == 'android_app_bundle': 1872 deps_proguard_enabled = [] 1873 deps_proguard_disabled = [] 1874 for d in deps.Direct('android_app_bundle_module'): 1875 if not d['device_classpath']: 1876 # We don't care about modules that have no Java code for proguarding. 1877 continue 1878 if d['proguard_enabled']: 1879 deps_proguard_enabled.append(d['name']) 1880 else: 1881 deps_proguard_disabled.append(d['name']) 1882 if deps_proguard_enabled and deps_proguard_disabled: 1883 raise Exception('Deps %s have proguard enabled while deps %s have ' 1884 'proguard disabled' % (deps_proguard_enabled, 1885 deps_proguard_disabled)) 1886 deps_info['proguard_enabled'] = bool(options.proguard_enabled) 1887 1888 if options.proguard_mapping_path: 1889 deps_info['proguard_mapping_path'] = options.proguard_mapping_path 1890 1891 # The java code for an instrumentation test apk is assembled differently for 1892 # ProGuard vs. non-ProGuard. 1893 # 1894 # Without ProGuard: Each library's jar is dexed separately and then combined 1895 # into a single classes.dex. A test apk will include all dex files not already 1896 # present in the apk-under-test. At runtime all test code lives in the test 1897 # apk, and the program code lives in the apk-under-test. 1898 # 1899 # With ProGuard: Each library's .jar file is fed into ProGuard, which outputs 1900 # a single .jar, which is then dexed into a classes.dex. A test apk includes 1901 # all jar files from the program and the tests because having them separate 1902 # doesn't work with ProGuard's whole-program optimizations. Although the 1903 # apk-under-test still has all of its code in its classes.dex, none of it is 1904 # used at runtime because the copy of it within the test apk takes precidence. 1905 1906 if options.type == 'android_apk' and options.tested_apk_config: 1907 if tested_apk_config['proguard_enabled']: 1908 assert options.proguard_enabled, ('proguard must be enabled for ' 1909 'instrumentation apks if it\'s enabled for the tested apk.') 1910 # Mutating lists, so no need to explicitly re-assign to dict. 1911 proguard_configs.extend( 1912 p for p in tested_apk_config['proguard_all_configs']) 1913 extra_proguard_classpath_jars.extend( 1914 p for p in tested_apk_config['proguard_classpath_jars']) 1915 tested_apk_config = GetDepConfig(options.tested_apk_config) 1916 deps_info['proguard_under_test_mapping'] = ( 1917 tested_apk_config['proguard_mapping_path']) 1918 elif options.proguard_enabled: 1919 # Not sure why you'd want to proguard the test apk when the under-test apk 1920 # is not proguarded, but it's easy enough to support. 1921 deps_info['proguard_under_test_mapping'] = '' 1922 1923 # Add all tested classes to the test's classpath to ensure that the test's 1924 # java code is a superset of the tested apk's java code 1925 device_classpath_extended = list(device_classpath) 1926 device_classpath_extended.extend( 1927 p for p in tested_apk_config['device_classpath'] 1928 if p not in device_classpath) 1929 # Include in the classpath classes that are added directly to the apk under 1930 # test (those that are not a part of a java_library). 1931 javac_classpath.add(tested_apk_config['unprocessed_jar_path']) 1932 javac_interface_classpath.add(tested_apk_config['interface_jar_path']) 1933 javac_full_classpath.add(tested_apk_config['unprocessed_jar_path']) 1934 javac_full_interface_classpath.add(tested_apk_config['interface_jar_path']) 1935 javac_full_classpath.update(tested_apk_config['javac_full_classpath']) 1936 javac_full_interface_classpath.update( 1937 tested_apk_config['javac_full_interface_classpath']) 1938 1939 # Exclude .jar files from the test apk that exist within the apk under test. 1940 tested_apk_library_deps = tested_apk_deps.All('java_library') 1941 tested_apk_dex_files = {c['dex_path'] for c in tested_apk_library_deps} 1942 all_dex_files = [p for p in all_dex_files if p not in tested_apk_dex_files] 1943 tested_apk_jar_files = set(tested_apk_config['device_classpath']) 1944 device_classpath = [ 1945 p for p in device_classpath if p not in tested_apk_jar_files 1946 ] 1947 1948 if options.type in ('android_apk', 'dist_aar', 'dist_jar', 1949 'android_app_bundle_module', 'android_app_bundle'): 1950 deps_info['proguard_all_configs'] = sorted(set(proguard_configs)) 1951 deps_info['proguard_classpath_jars'] = sorted( 1952 set(extra_proguard_classpath_jars)) 1953 1954 if options.final_dex_path: 1955 config['final_dex'] = {'path': options.final_dex_path} 1956 if is_apk_or_module_target or options.type == 'dist_jar': 1957 # Dependencies for the final dex file of an apk. 1958 deps_info['all_dex_files'] = all_dex_files 1959 1960 if is_java_target: 1961 config['javac']['classpath'] = sorted(javac_classpath) 1962 config['javac']['interface_classpath'] = sorted(javac_interface_classpath) 1963 # Direct() will be of type 'java_annotation_processor', and so not included 1964 # in All('java_library'). 1965 # Annotation processors run as part of the build, so need host_jar_path. 1966 config['javac']['processor_classpath'] = [ 1967 c['host_jar_path'] for c in processor_deps.Direct() 1968 if c.get('host_jar_path') 1969 ] 1970 config['javac']['processor_classpath'] += [ 1971 c['host_jar_path'] for c in processor_deps.All('java_library') 1972 ] 1973 config['javac']['processor_classes'] = sorted( 1974 c['main_class'] for c in processor_deps.Direct()) 1975 deps_info['javac_full_classpath'] = list(javac_full_classpath) 1976 deps_info['javac_full_interface_classpath'] = list( 1977 javac_full_interface_classpath) 1978 elif options.type == 'android_app_bundle': 1979 # bundles require javac_full_classpath to create .aab.jar.info and require 1980 # javac_full_interface_classpath for lint. 1981 javac_full_classpath = OrderedSet() 1982 javac_full_interface_classpath = OrderedSet() 1983 for d in deps.Direct('android_app_bundle_module'): 1984 javac_full_classpath.update(d['javac_full_classpath']) 1985 javac_full_interface_classpath.update(d['javac_full_interface_classpath']) 1986 javac_full_classpath.add(d['unprocessed_jar_path']) 1987 javac_full_interface_classpath.add(d['interface_jar_path']) 1988 deps_info['javac_full_classpath'] = list(javac_full_classpath) 1989 deps_info['javac_full_interface_classpath'] = list( 1990 javac_full_interface_classpath) 1991 1992 if options.type in ('android_apk', 'android_app_bundle', 1993 'android_app_bundle_module', 'dist_aar', 'dist_jar'): 1994 deps_info['device_classpath'] = device_classpath 1995 if options.trace_events_jar_dir: 1996 trace_event_rewritten_device_classpath = [] 1997 for jar_path in device_classpath: 1998 file_path = jar_path.replace('../', '') 1999 file_path = file_path.replace('obj/', '') 2000 file_path = file_path.replace('gen/', '') 2001 file_path = file_path.replace('.jar', '.tracing_rewritten.jar') 2002 rewritten_jar_path = os.path.join(options.trace_events_jar_dir, 2003 file_path) 2004 trace_event_rewritten_device_classpath.append(rewritten_jar_path) 2005 2006 deps_info['trace_event_rewritten_device_classpath'] = ( 2007 trace_event_rewritten_device_classpath) 2008 2009 if options.tested_apk_config: 2010 deps_info['device_classpath_extended'] = device_classpath_extended 2011 2012 if options.type in ('android_apk', 'dist_jar'): 2013 all_interface_jars = [] 2014 if options.interface_jar_path: 2015 all_interface_jars.append(options.interface_jar_path) 2016 all_interface_jars.extend(c['interface_jar_path'] for c in all_library_deps) 2017 2018 config['dist_jar'] = { 2019 'all_interface_jars': all_interface_jars, 2020 } 2021 2022 if is_apk_or_module_target: 2023 manifest = AndroidManifest(options.android_manifest) 2024 deps_info['package_name'] = manifest.GetPackageName() 2025 if not options.tested_apk_config and manifest.GetInstrumentationElements(): 2026 # This must then have instrumentation only for itself. 2027 manifest.CheckInstrumentationElements(manifest.GetPackageName()) 2028 2029 library_paths = [] 2030 java_libraries_list = None 2031 if options.shared_libraries_runtime_deps: 2032 library_paths = _ExtractSharedLibsFromRuntimeDeps( 2033 options.shared_libraries_runtime_deps) 2034 java_libraries_list = _CreateJavaLibrariesList(library_paths) 2035 all_inputs.append(options.shared_libraries_runtime_deps) 2036 2037 secondary_abi_library_paths = [] 2038 if options.secondary_abi_shared_libraries_runtime_deps: 2039 secondary_abi_library_paths = _ExtractSharedLibsFromRuntimeDeps( 2040 options.secondary_abi_shared_libraries_runtime_deps) 2041 secondary_abi_library_paths.sort() 2042 paths_without_parent_dirs = [ 2043 p for p in secondary_abi_library_paths if os.path.sep not in p 2044 ] 2045 if paths_without_parent_dirs: 2046 sys.stderr.write('Found secondary native libraries from primary ' 2047 'toolchain directory. This is a bug!\n') 2048 sys.stderr.write('\n'.join(paths_without_parent_dirs)) 2049 sys.stderr.write('\n\nIt may be helpful to run: \n') 2050 sys.stderr.write(' gn path out/Default //chrome/android:' 2051 'monochrome_secondary_abi_lib //base:base\n') 2052 sys.exit(1) 2053 2054 all_inputs.append(options.secondary_abi_shared_libraries_runtime_deps) 2055 2056 native_library_placeholder_paths = action_helpers.parse_gn_list( 2057 options.native_lib_placeholders) 2058 native_library_placeholder_paths.sort() 2059 2060 secondary_native_library_placeholder_paths = action_helpers.parse_gn_list( 2061 options.secondary_native_lib_placeholders) 2062 secondary_native_library_placeholder_paths.sort() 2063 2064 loadable_modules = action_helpers.parse_gn_list(options.loadable_modules) 2065 loadable_modules.sort() 2066 secondary_abi_loadable_modules = action_helpers.parse_gn_list( 2067 options.secondary_abi_loadable_modules) 2068 secondary_abi_loadable_modules.sort() 2069 2070 config['native'] = { 2071 'libraries': 2072 library_paths, 2073 'native_library_placeholders': 2074 native_library_placeholder_paths, 2075 'secondary_abi_libraries': 2076 secondary_abi_library_paths, 2077 'secondary_native_library_placeholders': 2078 secondary_native_library_placeholder_paths, 2079 'java_libraries_list': 2080 java_libraries_list, 2081 'library_always_compress': 2082 options.library_always_compress, 2083 'loadable_modules': 2084 loadable_modules, 2085 'secondary_abi_loadable_modules': 2086 secondary_abi_loadable_modules, 2087 } 2088 2089 # Collect java resources 2090 java_resources_jars = [d['java_resources_jar'] for d in all_library_deps 2091 if 'java_resources_jar' in d] 2092 if options.tested_apk_config: 2093 tested_apk_resource_jars = [d['java_resources_jar'] 2094 for d in tested_apk_library_deps 2095 if 'java_resources_jar' in d] 2096 java_resources_jars = [jar for jar in java_resources_jars 2097 if jar not in tested_apk_resource_jars] 2098 java_resources_jars.sort() 2099 config['java_resources_jars'] = java_resources_jars 2100 2101 if is_apk_or_module_target or options.type == 'robolectric_binary': 2102 # android_resources deps which had recursive_resource_deps set should not 2103 # have the manifests from the recursively collected deps added to this 2104 # module. This keeps the manifest declarations in the child DFMs, since they 2105 # will have the Java implementations. 2106 def ExcludeRecursiveResourcesDeps(config): 2107 return not config.get('includes_recursive_resources', False) 2108 2109 extra_manifest_deps = [ 2110 GetDepConfig(p) for p in GetAllDepsConfigsInOrder( 2111 deps_configs_paths, filter_func=ExcludeRecursiveResourcesDeps) 2112 ] 2113 # Manifests are listed from highest priority to lowest priority. 2114 # Ensure directly manfifests come first, and then sort the rest by name. 2115 # https://developer.android.com/build/manage-manifests#merge_priorities 2116 config['extra_android_manifests'] = list(mergeable_android_manifests) 2117 manifests_from_deps = [] 2118 for c in extra_manifest_deps: 2119 manifests_from_deps += c.get('mergeable_android_manifests', []) 2120 manifests_from_deps.sort(key=lambda p: (os.path.basename(p), p)) 2121 config['extra_android_manifests'] += manifests_from_deps 2122 2123 config['assets'], config['uncompressed_assets'], locale_paks = ( 2124 _MergeAssets(deps.All('android_assets'))) 2125 deps_info['locales_java_list'] = _CreateJavaLocaleListFromAssets( 2126 config['uncompressed_assets'], locale_paks) 2127 2128 if options.java_resources_jar_path: 2129 deps_info['java_resources_jar'] = options.java_resources_jar_path 2130 2131 # DYNAMIC FEATURE MODULES: 2132 # There are two approaches to dealing with modules dependencies: 2133 # 1) Perform steps in android_apk_or_module(), with only the knowledge of 2134 # ancesstor splits. Our implementation currently allows only for 2 levels: 2135 # base -> parent -> leaf 2136 # Bundletool normally fails if two leaf nodes merge the same manifest or 2137 # resources. The fix is to add the common dep to the chrome or base module 2138 # so that our deduplication logic will work. 2139 # RemoveObjDups() implements this approach. 2140 # 2) Perform steps in android_app_bundle(), with knowledge of full set of 2141 # modules. This is required for dex because it can handle the case of two 2142 # leaf nodes having the same dep, and promoting that dep to their common 2143 # parent. 2144 # _DedupFeatureModuleSharedCode() implements this approach. 2145 if base_module_build_config: 2146 ancestors = [base_module_build_config] 2147 if parent_module_build_config is not base_module_build_config: 2148 ancestors += [parent_module_build_config] 2149 for ancestor in ancestors: 2150 RemoveObjDups(config, ancestor, 'deps_info', 'dependency_zips') 2151 RemoveObjDups(config, ancestor, 'deps_info', 'dependency_zip_overlays') 2152 RemoveObjDups(config, ancestor, 'deps_info', 'extra_package_names') 2153 RemoveObjDups(config, ancestor, 'extra_android_manifests') 2154 2155 if is_java_target: 2156 jar_to_target = {} 2157 _AddJarMapping(jar_to_target, [deps_info]) 2158 _AddJarMapping(jar_to_target, all_deps) 2159 if base_module_build_config: 2160 _AddJarMapping(jar_to_target, [base_module_build_config['deps_info']]) 2161 if parent_module_build_config is not base_module_build_config: 2162 _AddJarMapping(jar_to_target, [parent_module_build_config['deps_info']]) 2163 if options.tested_apk_config: 2164 _AddJarMapping(jar_to_target, [tested_apk_config]) 2165 for jar, target in zip(tested_apk_config['javac_full_classpath'], 2166 tested_apk_config['javac_full_classpath_targets']): 2167 jar_to_target[jar] = target 2168 2169 # Used by bytecode_processor to give better error message when missing 2170 # deps are found. Both javac_full_classpath_targets and javac_full_classpath 2171 # must be in identical orders, as they get passed as separate arrays and 2172 # then paired up based on index. 2173 config['deps_info']['javac_full_classpath_targets'] = [ 2174 jar_to_target[x] for x in deps_info['javac_full_classpath'] 2175 ] 2176 2177 build_utils.WriteJson(config, options.build_config, only_if_changed=True) 2178 2179 if options.depfile: 2180 action_helpers.write_depfile(options.depfile, options.build_config, 2181 sorted(set(all_inputs))) 2182 2183 if options.store_deps_for_debugging_to: 2184 GetDepConfig(options.build_config) # Add it to cache. 2185 _CopyBuildConfigsForDebugging(options.store_deps_for_debugging_to) 2186 2187 return 0 2188 2189 2190if __name__ == '__main__': 2191 sys.exit(main(sys.argv[1:])) 2192