Lines Matching refs:Python
4 Pigweed's GN Python Build
14 Pigweed uses a custom GN-based build system to manage its Python code. The
15 Pigweed Python build supports packaging, installation and distribution of
16 interdependent local Python packages. It also provides for fast, incremental
20 Pigweed's Python code is exclusively managed by GN, but the GN-based build may
22 setup uses GN to set up the initial Python environment, regardless of the final
23 build system. As needed, non-GN projects can declare just their Python packages
28 In addition to compiler commands a Pigweed GN build will execute Python scripts
32 ``python``. Running Python on it's own by default will make any Python packages
35 workstation. To get around this the Python community uses `virtual environments
37 set of Python packages separate from the host system.
41 graph. Once created, all required third-party Python packages needed for the
64 subgraph pyactions1[Python venv 1]
73 subgraph pyactions2[Python venv 2]
92 dependencies you may be wondering how you can import your own in-tree Python
93 packages. Python code run in the build may still import any in-tree Python
96 provided. Having that Python dependency defined in GN allows the
101 dependency for any Python action or package is missing.
103 .. admonition:: Benefits of Python ``venvs`` in GN
106 - Using venvs to execute Python in GN provides reproducible builds with fixed
108 - Using ``PYTHONPATH`` coupled with ``python_deps`` to import in-tree Python
112 Managing Python Requirements
117 Build Time Python Virtualenv
119 Pigweed's GN Python build infrastructure relies on `Python virtual environments
120 <https://docs.python.org/3/library/venv.html>`_ for executing Python code. This
122 Python constraints where all Python tests, linting and
125 There must be at least one venv for Python defined in GN. There can be multiple
138 optional ``venv`` argument to specify which Python venv it should run
144 Third-party Python Requirements and Constraints
146 Your project may have third party Python dependencies you wish to install into
148 to add Python package dependencies:
154 if your in-tree Python package requires an external Python package.
156 2. Create a standard Python ``requirements.txt`` file in your project and add it
179 Every project should ideally inherit Pigweed's third party Python package
180 version. This is accomplished via `Python constraints files
183 installed. To inherit Pigweed's Python constraints include ``constraint.list``
281 single Python requirements file for replicating this ``pw_python_venv``
283 exact versions of each required Python package.
288 producing a self contained zip of in-tree and third party Python packages.
299 # This file is autogenerated by pip-compile with Python 3.11
325 Caching Python Packages for Offline Installation
333 that will download all Python packages from remote servers into a local
372 Note the above output has both Python wheel ``.whl`` and source distribution
373 ``.tar.gz`` files. The ``.whl`` may contain Python packages with precompiled C
380 If you need to cache Python packages for multiple platforms the
381 ``.vendor_wheels`` target will need to be run for each combination of Python
393 Plus all of the above duplicated for Python 3.10 and 3.9 (``cp310`` and
408 platform, architecture and Python version. This can take a significant amount of
416 The set of Python packages that will be downloaded is determined by the
418 current host OS and Python version. `pip-tools
452 # List of paths to folders containing Python wheels (*.whl) or source tar
461 # Python packages. On most systems this is located in ~/.cache/pip/
501 GN File Structure for Python Code
503 Here is a full example of what is required to build Python packages using
506 on the structure of Python packages.
551 # Inherit Pigweed Python constraints
617 # This group is built during bootstrap to setup the interactive Python
626 # In-tree Python packages
631 # Pigweed Python packages to include
641 # The default venv for Python actions in GN
652 # Ensure all third party Python dependencies are installed into this venv.
659 # single super Python package for installation into the bootstrapped virtual
671 # Install the project-tools super Python package into the bootstrapped
672 # Python venv.
679 Pigweed Module Structure for Python Code
681 Pigweed Python code is structured into standard Python packages. This makes it
682 simple to package and distribute Pigweed Python packages with common Python
685 Like all Pigweed source code, Python packages are organized into Pigweed
686 modules. A module's Python package is nested under a ``py/`` directory (see
690 :caption: :octicon:`file-directory;1em` Example layout of a Pigweed Python package.
710 check in ensures that all Python files are listed in a ``BUILD.gn``.
712 Pigweed prefers to define Python packages using ``setup.cfg`` files. In the
725 used for the given Python package. In Pigweed's case it always specifies using
737 The key abstraction in the Python build is the ``pw_python_package``.
738 A ``pw_python_package`` represents a Python package as a GN target. It is
750 subtarget represents different functionality in the Python build.
752 - ``<name>`` - Represents the Python files in the build, but does not take any
758 - ``<name>.lint`` - Runs static analysis tools on the Python code. This is a
761 - ``<name>.lint.mypy`` - Runs Mypy on all Python files, if enabled.
762 - ``<name>.lint.pylint`` - Runs Pylint on all Python files, if enabled.
763 - ``<name>.lint.ruff`` - Runs ruff on all Python files, if enabled.
765 - ``<name>.install`` - Installs the package in a Python virtual environment.
766 - ``<name>.wheel`` - Builds a Python wheel for this package.
768 To avoid unnecessary duplication, all Python actions are executed in the default
773 Tests for a Python package are listed in its ``pw_python_package`` target.
775 Python package. The build will run it when the test, the package, or one of its
806 Building Python wheels
809 distributing Python packages. The Pigweed Python build supports creating wheels
818 Wheels for a Python package and its transitive dependencies can be collected
825 target (see :ref:`module-pw_protobuf_compiler`). Python protobuf modules are
826 generated as standalone Python packages by default. Protocol buffers may also be
827 nested within existing Python packages. In this case, the Python package in the
828 source tree is incomplete; the final Python package, including protobufs, is
855 Python is an interpreted language, but it shares most build automation concerns
856 with other languages. Pigweed uses Python extensively and must address these
861 The Python programming langauge does not have an official build automation
862 system. However, there are numerous Python-focused build automation tools with
863 varying degrees of adoption. See the `Python Wiki
866 A few Python tools have become defacto standards, including `setuptools
869 These essential tools address key aspects of Python packaging and distribution,
872 provide more general build automation for Python.
875 Python and other languages used by Pigweed, including protocol buffers.
879 Pigweed's use of Python is different from many other projects. Pigweed is a
883 This section describes Python build automation challenges encountered by
888 Pigweed is organized into distinct modules. In Python, each module is a separate
892 The basic Python packaging tools lack dependency tracking for local packages.
911 There are lots of great Python libraries for testing, such as
914 write and execute individual Python tests, but are not well suited for managing
927 Various static analysis tools exist for Python. Two widely used, powerful tools
952 The protobuf compiler ``protoc`` generates Python modules from ``.proto`` files.
963 Python build.
968 - Correctly manage interdependent local Python packages.
977 Existing Python tools may be effective for Python codebases, but their utility
983 natively support Python, adding support is straightforward with GN templates.
987 example, C++, Go, and Python targets can depend on the same protobuf
993 clean syntax. This makes it feasible to use GN only for Python while building
996 Given these considerations, GN is an ideal choice for Pigweed's Python build.