1.. _module-pw_env_setup: 2 3------------ 4pw_env_setup 5------------ 6A classic problem in the embedded space is reducing the time from git clone 7to having a binary executing on a device. The issue is that an entire suite 8of tools is needed for non-trivial production embedded projects. For example: 9 10- A C++ compiler for your target device, and also for your host 11- A build system or three; for example, GN, Ninja, CMake, Bazel 12- A code formatting program like clang-format 13- A debugger like OpenOCD to flash and debug your embedded device (OpenOCD 14 support removed for Windows) 15- A known Python version with known modules installed for scripting 16- A Go compiler for the Go-based command line tools 17 18...and so on 19 20In the server space, container solutions like Docker or Podman solve this; 21however, in our experience container solutions are a mixed bag for embedded 22systems development where one frequently needs access to native system 23resources like USB devices, or must operate on Windows. 24 25``pw_env_setup`` is our compromise solution for this problem that works on Mac, 26Windows, and Linux. It leverages the Chrome packaging system `CIPD`_ to 27bootstrap a Python installation, which in turn inflates a virtual 28environment. The tooling is installed into your workspace, and makes no 29changes to your system. This tooling is designed to be reused by any 30project. 31 32.. _CIPD: https://github.com/luci/luci-go/tree/HEAD/cipd 33 34Users interact with ``pw_env_setup`` with two commands: ``. bootstrap.sh`` and 35``. activate.sh``. The bootstrap command always pulls down the current versions 36of CIPD packages and sets up the Python virtual environment. The activate 37command reinitializes a previously configured environment, and if none is found, 38runs bootstrap. 39 40.. note:: 41 42 On Windows the scripts used to set up the environment are ``bootstrap.bat`` 43 and ``activate.bat``. 44 45 ``bootstrap.fish`` and ``activate.fish`` are also available for `Fish shell 46 <https://fishshell.com/>`_ users. 47 48 For simplicity they will be referred to with the ``.sh`` endings unless the 49 distinction is relevant. 50 51On POSIX systems, the environment can be deactivated by running ``deactivate``. 52 53================================== 54Using pw_env_setup in your project 55================================== 56 57Downstream Projects Using Pigweed's Packages 58******************************************** 59 60Projects using Pigweed can leverage ``pw_env_setup`` to install Pigweed's 61dependencies or their own dependencies. Projects that only want to use Pigweed's 62dependencies without modifying them can just source Pigweed's ``bootstrap.sh`` 63and ``activate.sh`` scripts. 64 65An example of what your project's `bootstrap.sh` could look like is below. This 66assumes `bootstrap.sh` is at the top level of your repository. 67 68.. code-block:: bash 69 70 # Do not include a "#!" line, this must be sourced and not executed. 71 72 # This assumes the user is sourcing this file from it's parent directory. See 73 # below for a more flexible way to handle this. 74 PROJ_SETUP_SCRIPT_PATH="$(pwd)/bootstrap.sh" 75 76 export PW_PROJECT_ROOT="$(_python_abspath "$(dirname "$PROJ_SETUP_SCRIPT_PATH")")" 77 78 # You may wish to check if the user is attempting to execute this script 79 # instead of sourcing it. See below for an example of how to handle that 80 # situation. 81 82 # Source Pigweed's bootstrap utility script. 83 # Using '.' instead of 'source' for POSIX compatibility. Since users don't use 84 # dash directly, using 'source' in most documentation so users don't get 85 # confused and try to `./bootstrap.sh`. 86 . "$PW_PROJECT_ROOT/third_party/pigweed/pw_env_setup/util.sh" 87 88 pw_check_root "$PW_ROOT" 89 _PW_ACTUAL_ENVIRONMENT_ROOT="$(pw_get_env_root)" 90 export _PW_ACTUAL_ENVIRONMENT_ROOT 91 SETUP_SH="$_PW_ACTUAL_ENVIRONMENT_ROOT/activate.sh" 92 pw_bootstrap --args... # See below for details about args. 93 pw_finalize bootstrap "$SETUP_SH" 94 95 96Bazel Usage 97----------- 98It is possible to pull in a CIPD dependency into Bazel using WORKSPACE rules 99rather than using `bootstrap.sh`. e.g. 100 101.. code-block:: python 102 103 # WORKSPACE 104 105 load("//pw_env_setup/bazel/cipd_setup:cipd_rules.bzl", "pigweed_deps") 106 107 # Setup CIPD client and packages. 108 # Required by: pigweed. 109 # Used by modules: all. 110 pigweed_deps() 111 112 load("@cipd_deps//:cipd_init.bzl", "cipd_init") 113 114 cipd_init() 115 116 117This will make the entire set of Pigweeds remote repositories available to your 118project. Though these repositories will only be donwloaded if you use them. To 119get a full list of the remote repositories that this configures, run: 120 121.. code-block:: sh 122 123 bazel query //external:all | grep cipd_ 124 125All files and executables in each CIPD remote repository is exported and visible 126either directely (`@cipd_<dep>//:<file>`) or from 'all' filegroup 127(`@cipd_<dep>//:all`). 128 129From here it is possible to get access to the Bloaty binaries using the 130following command. For example; 131 132.. code-block:: sh 133 134 bazel run @cipd_pigweed_third_party_bloaty_embedded_linux_amd64//:bloaty \ 135 -- --help 136 137User-Friendliness 138----------------- 139 140You may wish to allow sourcing `bootstrap.sh` from a different directory. In 141that case you'll need the following at the top of `bootstrap.sh`. 142 143.. code-block:: bash 144 145 _python_abspath () { 146 python -c "import os.path; print(os.path.abspath('$@'))" 147 } 148 149 # Use this code from Pigweed's bootstrap to find the path to this script when 150 # sourced. This should work with common shells. PW_CHECKOUT_ROOT is only used in 151 # presubmit tests with strange setups, and can be omitted if you're not using 152 # Pigweed's automated testing infrastructure. 153 if test -n "$PW_CHECKOUT_ROOT"; then 154 PROJ_SETUP_SCRIPT_PATH="$(_python_abspath "$PW_CHECKOUT_ROOT/bootstrap.sh")" 155 unset PW_CHECKOUT_ROOT 156 # Shell: bash. 157 elif test -n "$BASH"; then 158 PROJ_SETUP_SCRIPT_PATH="$(_python_abspath "$BASH_SOURCE")" 159 # Shell: zsh. 160 elif test -n "$ZSH_NAME"; then 161 PROJ_SETUP_SCRIPT_PATH="$(_python_abspath "${(%):-%N}")" 162 # Shell: dash. 163 elif test ${0##*/} = dash; then 164 PROJ_SETUP_SCRIPT_PATH="$(_python_abspath \ 165 "$(lsof -p $$ -Fn0 | tail -1 | sed 's#^[^/]*##;')")" 166 # If everything else fails, try $0. It could work. 167 else 168 PROJ_SETUP_SCRIPT_PATH="$(_python_abspath "$0")" 169 fi 170 171You may also wish to check if the user is attempting to execute `bootstrap.sh` 172instead of sourcing it. Executing `bootstrap.sh` would download everything 173required for the environment, but cannot modify the environment of the parent 174process. To check for this add the following. 175 176.. code-block:: bash 177 178 # Check if this file is being executed or sourced. 179 _pw_sourced=0 180 # If not running in Pigweed's automated testing infrastructure the 181 # SWARMING_BOT_ID check is unnecessary. 182 if [ -n "$SWARMING_BOT_ID" ]; then 183 # If set we're running on swarming and don't need this check. 184 _pw_sourced=1 185 elif [ -n "$ZSH_EVAL_CONTEXT" ]; then 186 case $ZSH_EVAL_CONTEXT in *:file) _pw_sourced=1;; esac 187 elif [ -n "$KSH_VERSION" ]; then 188 [ "$(cd $(dirname -- $0) && pwd -P)/$(basename -- $0)" != \ 189 "$(cd $(dirname -- ${.sh.file}) && pwd -P)/$(basename -- ${.sh.file})" ] \ 190 && _pw_sourced=1 191 elif [ -n "$BASH_VERSION" ]; then 192 (return 0 2>/dev/null) && _pw_sourced=1 193 else # All other shells: examine $0 for known shell binary filenames 194 # Detects `sh` and `dash`; add additional shell filenames as needed. 195 case ${0##*/} in sh|dash) _pw_sourced=1;; esac 196 fi 197 198 _pw_eval_sourced "$_pw_sourced" 199 200Downstream Projects Using Different Packages 201******************************************** 202Projects depending on Pigweed but using additional or different packages should 203copy the Pigweed `sample project`'s ``bootstrap.sh`` and ``pigweed.json`` and 204update the call to ``pw_bootstrap``. Search for "downstream" for other places 205that may require changes, like setting the ``PW_ROOT`` and ``PW_PROJECT_ROOT`` 206environment variables. Explanations of parts of ``pigweed.json`` are described 207here. 208 209.. _sample project: https://pigweed.googlesource.com/pigweed/sample_project/+/HEAD 210 211``pw.pw_env_setup.root_variable`` 212 Variable used to point to the root of the source tree. Optional, can always 213 use ``PW_PROJECT_ROOT`` instead. (That variable will be set regardless of 214 whether this is provided.) 215 216``pw.pw_env_setup.relative_pigweed_root`` 217 Location of the Pigweed submodule within the source tree. Optional—environment 218 setup will work correctly without this. If present, will confirm that it's 219 correct. May be used by other tooling. 220 221``pw.pw_env_setup.cipd_package_files`` 222 CIPD package file. JSON file consisting of a list of additional CIPD package 223 files to import and a list of dictionaries with "path", "platforms", "subdir", 224 "tags", and "version_file" keys. Both top-level lists are optional. An 225 example is below. Only "path", "platforms", and "tags" are required. If 226 "version_file" is specified then ``pw doctor`` will fail if that version file 227 is not present. If "subdir" is specified then this packages will be installed 228 in a subdirectory of the directory created for packages in this file. 229 230.. code-block:: json 231 232 { 233 "included_files": [ 234 "foo.json" 235 ], 236 "packages": [ 237 { 238 "path": "infra/3pp/tools/go/${platform}", 239 "platforms": [ 240 "linux-amd64", 241 "linux-arm64", 242 "mac-amd64", 243 "windows-amd64" 244 ], 245 "subdir": "pa/th", 246 "tags": [ 247 "version:[email protected]" 248 ], 249 "version_file": ".versions/go.cipd_version" 250 } 251 ] 252 } 253 254``pw.pw_env_setup.project_actions`` 255 A list of plugins to load and run after CIPD setup, but prior to virtualenv 256 setup, for e.g. downloading project-specific tools or artifacts needed by 257 later steps. Particularly useful for downstream projects with limited CIPD 258 access. 259 260 A plugin is specified as a dictionary with two keys: "import_path" and 261 "module_name". The "import_path" is relative to the root of the checkout. 262 263 The specified module must provide a "run_actions" method which takes a single 264 argument, "env_vars", which is a pw_env_setup.Environment instance. 265 266 Sample plugin and pigweed.json blob: 267 268.. code-block:: python 269 270 """Sample pw_env_setup project action plugin. 271 272 A sample/starter project action plugin template for pw_env_setup. 273 """ 274 def run_action(**kwargs): 275 """Sample project action.""" 276 if "env" not in kwargs: 277 raise ValueError(f"Missing required kwarg 'env', got %{kwargs}") 278 279 kwargs["env"].prepend("PATH", "PATH_TO_NEW_TOOLS") 280 raise NotImplementedError("Sample project action running!") 281 282.. code-block:: json 283 284 "project_actions" : [ 285 { 286 "import_path": "pw_env_setup", 287 "module_name": "sample_project_action" 288 } 289 ], 290 291``pw.pw_env_setup.virtualenv.gn_args`` 292 Any necessary GN args to be used when installing Python packages. 293 294``pw.pw_env_setup.virtualenv.gn_targets`` 295 Target for installing Python packages. Downstream projects will need to 296 create targets to install their packages or only use Pigweed Python packages. 297 298``pw.pw_env_setup.virtualenv.gn_root`` 299 The root directory of your GN build tree, relative to ``PW_PROJECT_ROOT``. 300 This is the directory your project's ``.gn`` file is located in. If you're 301 only installing Pigweed Python packages, use the location of the Pigweed 302 submodule. 303 304``pw.pw_env_setup.virtualenv.requirements`` 305 A list of Python Pip requirements files for installing into the Pigweed 306 virtualenv. Each file will be passed as additional ``--requirement`` argument 307 to a single ```pip install`` at the beginning of bootstrap's ``Python 308 environment`` setup stage. See the `Requirements Files documentation`_ for 309 details on what can be specified using requirements files. 310 311``pw.pw_env_setup.virtualenv.constraints`` 312 A list of Python Pip constraints files. These constraints will be passed to 313 every ``pip`` invocation as an additional ``--constraint`` argument during 314 bootstrap. virtualenv. See the `Constraints Files documentation`_ for details 315 on formatting. 316 317``pw.pw_env_setup.virtualenv.system_packages`` 318 A boolean value that can be used the give the Python virtual environment 319 access to the system site packages. Defaults to ``false``. 320 321``pw.pw_env_setup.virtualenv.pip_install_offline`` 322 A boolean value that adds ``--no-index`` to all ``pip install`` commands that 323 are part of bootstrap. This forces pip to not reach out to the internet 324 (usually `pypi.org <https://pypi.org/>`_) to download packages. Using this 325 option requires setting 326 ``pw.pw_env_setup.virtualenv.pip_install_find_links``. Defaults to 327 ``false``. 328 329 .. seealso:: 330 The Python GN guide for offline pip installation: 331 :ref:`docs-python-build-installing-offline` 332 333``pw.pw_env_setup.virtualenv.pip_install_find_links`` 334 List of paths to folders containing Python wheels (``*.whl``) or source tar 335 files (``*.tar.gz``). Pip will check each of these directories when looking 336 for potential install candidates. Each path will be passed to all ``pip 337 install`` commands as ``--find-links PATH``. 338 339 .. tip:: 340 Environment variables may be used in these paths. For example: 341 342 .. code-block:: json 343 344 "virtualenv": { 345 "pip_install_find_links": [ 346 "${PW_PROJECT_ROOT}/pip_cache" 347 ] 348 } 349 350``pw.pw_env_setup.virtualenv.pip_install_require_hashes`` 351 Adds ``--require-hashes`` This option enforces hash checking on Python 352 package files. Defaults to ``false``. 353 354``pw.pw_env_setup.virtualenv.pip_install_disable_cache`` 355 A boolean value that adds ``--no-cache-dir`` to all ``pip install`` commands 356 that are part of bootstrap. This forces pip to ignore any previously cached 357 Python packages. On most systems this is located in 358 ``~/.cache/pip/``. Defaults to ``false``. 359 360``pw.pw_env_setup.optional_submodules`` 361 By default environment setup will check that all submodules are present in 362 the checkout. Any submodules in this list are excluded from that check. 363 364``pw.pw_env_setup.required_submodules`` 365 If this is specified instead of ``optional_submodules`` bootstrap will only 366 complain if one of the required submodules is not present. Combining this 367 with ``optional_submodules`` is not supported. 368 369``pw.pw_env_setup.pw_packages`` 370 A list of packages to install using :ref:`pw_package <module-pw_package>` 371 after the rest of bootstrap completes. 372 373``pw.pw_env_setup.gni_file`` 374 Location to write a ``.gni`` file containing paths to many things within the 375 environment directory. Defaults to 376 ``build_overrides/pigweed_environment.gni``. 377 378``pw.pw_env_setup.json_file`` 379 Location to write a ``.json`` file containing step-by-step modifications to 380 the environment, for reading by tools that don't inherit an environment from 381 a sourced ``bootstrap.sh``. 382 383``pw.pw_env_setup.rosetta`` 384 Whether to use Rosetta to use amd64 packages on arm64 Macs. Accepted values 385 are ``never``, ``allow``, and ``force``. For now, ``allow`` means ``force``. 386 At some point in the future ``allow`` will be changed to mean ``never``. 387 388An example of a config file is below. 389 390.. code-block:: json 391 392 { 393 "pw": { 394 "pw_env_setup": { 395 "root_variable": "EXAMPLE_ROOT", 396 "cipd_package_files": [ 397 "pigweed/pw_env_setup/py/pw_env_setup/cipd_setup/pigweed.json", 398 "pigweed/pw_env_setup/py/pw_env_setup/cipd_setup/luci.json" 399 "tools/myprojectname.json" 400 ], 401 "virtualenv": { 402 "gn_root": ".", 403 "gn_targets": [ 404 ":python.install", 405 ], 406 "system_packages": false 407 }, 408 "pw_packages": [], 409 "optional_submodules": [ 410 "optional/submodule/one", 411 "optional/submodule/two" 412 ], 413 "gni_file": "tools/environment.gni", 414 "json_file": "tools/environment.json", 415 "rosetta": "allow" 416 } 417 } 418 } 419 420Only the packages necessary for almost all projects based on Pigweed are 421included in the ``cipd_setup/pigweed.json`` file. A number of other files are 422present in that directory for projects that need more than the minimum. 423Internal-Google projects using LUCI should at least include ``luci.json``. 424 425In case the CIPD packages need to be referenced from other scripts, variables 426like ``PW_${BASENAME}_CIPD_INSTALL_DIR`` point to the CIPD install directories, 427where ``${BASENAME}`` is ``"PIGWEED"`` for 428``"pigweed/pw_env_setup/py/pw_env_setup/cipd_setup/pigweed.json"`` and 429``"LUCI"`` for 430``"pigweed/pw_env_setup/py/pw_env_setup/cipd_setup/luci.json"``. This example 431would set the following environment variables. 432 433- ``PW_LUCI_CIPD_INSTALL_DIR`` 434- ``PW_MYPROJECTNAME_CIPD_INSTALL_DIR`` 435- ``PW_PIGWEED_CIPD_INSTALL_DIR`` 436 437These directories are also referenced in the gni_file specified by the 438environment config file as ``dir_cipd_${BASENAME}``. This allows the GN build to 439reliably reference these directories without using GN ``getenv()`` calls or 440hardcoding paths. 441 442In addition, ``PW_${BASENAME}_CIPD_INSTALL_DIR`` and 443``PW_${BASENAME}_CIPD_INSTALL_DIR/bin`` are both added to ``PATH`` for each 444package directory. 445 446If multiple packages install executables with the same name, the file mentioned 447last topologically takes priority. For example, with the file contents below, 448``d.json``'s entries will appear in ``PATH`` before ``c.json``'s, which will 449appear before ``b.json``'s, which will appear before ``a.json``'s. 450 451.. code-block:: json 452 :caption: :octicon:`file;1em` pigweed.json 453 454 { 455 "pw": { 456 "pw_env_setup": { 457 "cipd_package_files": [ 458 "a.json", 459 "b.json", 460 "d.json" 461 ] 462 } 463 } 464 } 465 466.. code-block:: json 467 :caption: :octicon:`file;1em` a.json 468 469 { 470 "package_files": [ 471 // ... 472 ] 473 } 474 475.. code-block:: json 476 :caption: :octicon:`file;1em` b.json 477 478 { 479 "included_files": ["c.json"], 480 "package_files": [ 481 // ... 482 ] 483 } 484 485.. code-block:: json 486 :caption: :octicon:`file;1em` c.json 487 488 { 489 "package_files": [ 490 // ... 491 ] 492 } 493 494.. code-block:: json 495 :caption: :octicon:`file;1em` d.json 496 497 { 498 "package_files": [ 499 // ... 500 ] 501 } 502 503.. code-block:: 504 :caption: Effective File Loading Order 505 506 pigweed.json 507 a.json 508 b.json 509 c.json 510 d.json 511 512Pinning Python Packages 513*********************** 514Python modules usually express dependencies as ranges, which makes it easier to 515install many Python packages that might otherwise have conflicting dependencies. 516However, this means version of packages can often change underneath us and 517builds will not be hermetic. 518 519To ensure versions don't change without approval, Pigweed by default pins the 520versions of packages it depends on using a `pip constraints file`_. To pin the 521versions of additional packages your project depends on, run 522``pw python-packages list <path/to/constraints/file>`` and then add 523``pw_build_PIP_CONSTRAINTS = ["//path/to/constraints/file"]`` to your project's 524``.gn`` file (see `Pigweed's .gn file`_ for an example). 525 526.. _pip constraints file: https://pip.pypa.io/en/stable/user_guide/#constraints-files 527.. _default constraints: https://cs.pigweed.dev/pigweed/+/main:pw_env_setup/py/pw_env_setup/virtualenv_setup/constraint.list 528.. _Pigweed's .gn file: https://cs.pigweed.dev/pigweed/+/main:.gn 529 530To update packages, set ``pw_build_PIP_CONSTRAINTS = []``, delete the 531environment, and bootstrap again. Then run the ``list`` command from above 532again, and run ``pw presubmit``. 533 534Environment Variables 535********************* 536Input Variables 537--------------- 538The following environment variables affect env setup behavior. Most users will 539never need to set these. 540 541``CIPD_CACHE_DIR`` 542 Location of CIPD cache dir. Read by CIPD, but if unset will be defaulted to 543 ``$HOME/.cipd-cache-dir``. 544 545``PW_NO_CIPD_CACHE_DIR`` 546 Disables the CIPD cache. 547 548``PW_ACTIVATE_SKIP_CHECKS`` 549 If set, skip running ``pw doctor`` at end of bootstrap/activate. Intended to 550 be used by automated tools but not interactively. 551 552``PW_BANNER_FUNC`` 553 Command to print a banner at the beginning of bootstrap. 554 555``PW_BOOTSTRAP_PYTHON`` 556 Python executable to be used, for example "python3". Defaults to 557 "python3" if that's in ``PATH``, then tries "python". 558 559``PW_CIPD_SERVICE_ACCOUNT_JSON`` 560 Value to pass as ``-service-account-json`` to CIPD invocations. This should 561 point either to a service account JSON key file, or be the magical value 562 ``:gce`` to tell the tool to fetch tokens from GCE metadata server. 563 564``PW_ENVIRONMENT_ROOT`` 565 Location to which packages are installed. Defaults to ``environment`` folder 566 within the checkout root. This variable is cleared after environment setup is 567 complete. 568 569``PW_ENVSETUP_DISABLE_SPINNER`` 570 Disable the spinner during env setup. Intended to be used when the output is 571 being redirected to a log. 572 573``PW_ENVSETUP_DISABLE_SPINNER`` 574 Disable the console spinner that runs when waiting for env setup steps to 575 complete. 576 577``PW_ENVSETUP_NO_BANNER`` 578 Skip printing the banner. 579 580``PW_ENVSETUP_QUIET`` 581 Disables all non-error output. 582 583``PW_PROJECT_ROOT`` 584 The absolute path of the project using Pigweed's env setup. For Pigweed this 585 is the same as ``PW_ROOT``. This should be set by the project's bootstrap 586 script. 587 588``PW_ROOT`` 589 The absolute path to the Pigweed repository within ``PW_PROJECT_ROOT``. This 590 should be set by the project's bootstrap script. 591 592Output Variables 593---------------- 594The following environment variables are set by env setup. 595 596``PATH`` 597 System executable search path. Many of the environment variables below are 598 also added to this variable. 599 600``_PW_ACTUAL_ENVIRONMENT_ROOT`` 601 Location the environment was installed into. Separate from 602 ``PW_ENVIRONMENT_ROOT`` because setting that implicitly and switching to 603 another project directory causes unexpected behavior. 604 605``PW_CIPD_INSTALL_DIR`` 606 Top-level CIPD install directory. This is where the ``cipd`` executable is. 607 608``PW_*_CIPD_INSTALL_DIR`` 609 Each CIPD package file is installed into its own directory. This allows other 610 tools to determine what those directories are. The ``*`` is replaced with an 611 all-caps version of the basename of the package file, without the extension. 612 (E.g., "path/foo.json" becomes ``PW_FOO_CIPD_INSTALL_DIR``.) 613 614``PW_PACKAGE_ROOT`` 615 Location that packages installed by ``pw package`` will be installed to. 616 617``VIRTUAL_ENV`` 618 Path to Pigweed's virtualenv. 619 620Non-Shell Environments 621********************** 622If using this outside of bash—for example directly from an IDE or CI 623system—users can process the ``actions.json`` file that's generated in the 624location specified by the environment config. It lists variables to set, clear, 625and modify. An example ``actions.json`` is shown below. The "append" and 626"prepend" actions are listed in the order they should be applied, so the 627``<pigweed-root>/out/host/host_tools`` entry should be at the beginning of 628``PATH`` and not in the middle somewhere. 629 630.. code-block:: json 631 632 { 633 "modify": { 634 "PATH": { 635 "append": [], 636 "prepend": [ 637 "<pigweed-root>/environment/cipd", 638 "<pigweed-root>/environment/cipd/pigweed", 639 "<pigweed-root>/environment/cipd/pigweed/bin", 640 "<pigweed-root>/environment/cipd/luci", 641 "<pigweed-root>/environment/cipd/luci/bin", 642 "<pigweed-root>/environment/pigweed-venv/bin", 643 "<pigweed-root>/out/host/host_tools" 644 ], 645 "remove": [] 646 } 647 }, 648 "set": { 649 "PW_PROJECT_ROOT": "<pigweed-root>", 650 "PW_ROOT": "<pigweed-root>", 651 "_PW_ACTUAL_ENVIRONMENT_ROOT": "<pigweed-root>/environment", 652 "PW_CIPD_INSTALL_DIR": "<pigweed-root>/environment/cipd", 653 "CIPD_CACHE_DIR": "<home>/.cipd-cache-dir", 654 "PW_PIGWEED_CIPD_INSTALL_DIR": "<pigweed-root>/environment/cipd/pigweed", 655 "PW_LUCI_CIPD_INSTALL_DIR": "<pigweed-root>/environment/cipd/luci", 656 "VIRTUAL_ENV": "<pigweed-root>/environment/pigweed-venv", 657 "PYTHONHOME": null, 658 "__PYVENV_LAUNCHER__": null 659 } 660 } 661 662Many of these variables are directly exposed to the GN build as well, through 663the GNI file specified in the environment config file. 664 665.. code-block:: 666 667 declare_args() { 668 pw_env_setup_CIPD_LUCI = "<environment-root>/cipd/packages/luci" 669 pw_env_setup_CIPD_PIGWEED = "<environment-root>/cipd/packages/pigweed" 670 pw_env_setup_PACKAGE_ROOT = "<environment-root>/packages" 671 pw_env_setup_VIRTUAL_ENV = "<environment-root>/pigweed-venv" 672 } 673 674It's straightforward to use these variables. 675 676.. code-block:: cpp 677 678 import("//build_overrides/pigweed_environment.gni") 679 680 deps = [ "$pw_env_setup_CIPD_PIGWEED/..." ] 681 682Implementation 683************** 684The environment is set up by installing CIPD and Python packages in 685``PW_ENVIRONMENT_ROOT`` or ``<checkout>/environment``, and saving modifications 686to environment variables in setup scripts in those directories. To support 687multiple operating systems this is done in an operating system-agnostic manner 688and then written into operating system-specific files to be sourced now and in 689the future when running ``activate.sh`` instead of ``bootstrap.sh``. In the 690future these could be extended to C shell and PowerShell. A logical mapping of 691high-level commands to system-specific initialization files is shown below. 692 693.. grid:: 1 694 :padding: 0 695 696 .. grid-item-card:: 697 :columns: 12 698 :class-header: font-monospace 699 700 SET $PW_ROOT /home/$USER/pigweed 701 ^^^ 702 703 .. grid:: 2 704 :margin: 0 705 :padding: 0 706 707 .. grid-item:: **Windows** 708 709 .. grid-item:: **Linux & Mac (sh-compatible shells)** 710 711 .. grid:: 2 712 :margin: 0 713 :padding: 0 714 715 .. grid-item:: 716 717 .. code-block:: dosbatch 718 719 set PW_ROOT /home/%USER%/pigweed 720 721 .. grid-item:: 722 723 .. code-block:: shell 724 725 PW_ROOT="/home/$USER/pigweed" 726 export PW_ROOT 727 728.. grid:: 1 729 :padding: 0 730 731 .. grid-item-card:: 732 :columns: 12 733 :class-header: font-monospace 734 735 PREPEND $PATH $PW_ROOT/.env/bin 736 ^^^ 737 .. grid:: 2 738 :margin: 0 739 :padding: 0 740 741 .. grid-item:: **Windows** 742 743 .. grid-item:: **Linux & Mac (sh-compatible shells)** 744 745 .. grid:: 2 746 :margin: 0 747 :padding: 0 748 749 .. grid-item:: 750 751 .. code-block:: dosbatch 752 753 set PATH=%PW_ROOT%/.env/bin;%PATH% 754 755 .. grid-item:: 756 757 .. code-block:: shell 758 759 PATH="$(\ 760 echo "$PATH" | \ 761 sed "s|:$PW_ROOT/.env/bin:|:|g;" | \ 762 sed "s|^$PW_ROOT/.env/bin:||g;" | \ 763 sed "s|:$PW_ROOT/.env/bin$||g;")" 764 PATH="$PW_ROOT/.env/bin;$PATH" 765 export PATH 766 767.. grid:: 1 768 :padding: 0 769 770 .. grid-item-card:: 771 :columns: 12 772 :class-header: font-monospace 773 774 ECHO "Setup Complete!" 775 ^^^ 776 777 .. grid:: 2 778 :margin: 0 779 :padding: 0 780 781 .. grid-item:: **Windows** 782 783 .. grid-item:: **Linux & Mac (sh-compatible shells)** 784 785 786 .. grid:: 2 787 :margin: 0 788 :padding: 0 789 790 .. grid-item:: 791 792 .. code-block:: dosbatch 793 794 echo Setup Complete! 795 796 .. grid-item:: 797 798 .. code-block:: shell 799 800 echo "Setup Complete!" 801 802 803.. _Requirements Files documentation: https://pip.pypa.io/en/stable/user_guide/#requirements-files 804.. _Constraints Files documentation: https://pip.pypa.io/en/stable/user_guide/#constraints-files 805