1.. _docs-module-structure: 2 3---------------- 4Module Structure 5---------------- 6The Pigweed :ref:`module <docs-glossary-module>` structure is designed to keep 7as much code as possible for a particular slice of functionality in one place. 8That means including the code from multiple languages, as well as all the 9related documentation and tests. 10 11Additionally, the structure is designed to limit the number of places a file 12could go, so that when reading callsites it is obvious where a header is from. 13That is where the duplicated ``<module>`` occurrences in file paths comes from. 14 15Example module structure 16------------------------ 17.. code-block:: text 18 19 pw_foo/... 20 21 docs.rst # Docs landing page (required) 22 concepts.rst # Conceptual docs (optional) 23 design.rst # Design docs (optional) 24 guides.rst # How-to guides (optional) 25 api.rst # API reference (optional) 26 cli.rst # CLI reference (optional) 27 gui.rst # GUI reference (optional) 28 tutorials/*.rst # Tutorials (optional) 29 30 BUILD.gn # GN build required 31 BUILD # Bazel build required 32 33 # C++ public headers; the repeated module name is required 34 public/pw_foo/foo.h 35 public/pw_foo/baz.h 36 37 # Exposed private headers go under internal/ 38 public/pw_foo/internal/bar.h 39 public/pw_foo/internal/qux.h 40 41 # Public override headers must go in 'public_overrides' 42 public_overrides/gtest/gtest.h 43 public_overrides/string.h 44 45 # Private headers go into <module>_*/... 46 pw_foo_internal/zap.h 47 pw_foo_private/zip.h 48 pw_foo_secret/alxx.h 49 50 # C++ implementations go in the root 51 foo_impl.cc 52 foo.cc 53 baz.cc 54 bar.cc 55 zap.cc 56 zip.cc 57 alxx.cc 58 59 # C++ tests also go in the root 60 foo_test.cc 61 bar_test.cc 62 zip_test.cc 63 64 # Python files go into 'py/<module>/...' 65 py/BUILD.gn # Python packages are declared in GN using pw_python_package 66 py/setup.py # Python files are structured as standard Python packages 67 py/foo_test.py # Tests go in py/ but outside of the Python package 68 py/bar_test.py 69 py/pw_foo/__init__.py 70 py/pw_foo/__main__.py 71 py/pw_foo/bar.py 72 py/pw_foo/py.typed # Indicates that this package has type annotations 73 74 # Rust crates go into 'rust/...' 75 rust/BUILD.bazel 76 rust/crate_one.rs # Single file crates are in rust/<crate_name>.rs 77 rust/crate_two/lib.rs # Multi-file crate's top level source in: 78 # rust/<crate>/lib.rs 79 rust/crate_two/mod_one.rs # Multi-file crate's modules in: 80 rust/crate_two/mod_two.rs # rust/<crate>/<module_name>.rs 81 # Prefer not using mod.rs files. 82 83 # Go files go into 'go/...' 84 go/... 85 86 # Examples go in examples/, mixing different languages 87 examples/demo.py 88 examples/demo.cc 89 examples/demo.go 90 examples/BUILD.gn 91 examples/BUILD 92 93 # Size reports go under size_report/ 94 size_report/BUILD.gn 95 size_report/base.cc 96 size_report/use_case_a.cc 97 size_report/use_case_b.cc 98 99 # Protobuf definition files go into <module>_protos/... 100 pw_foo_protos/foo.proto 101 pw_foo_protos/internal/zap.proto 102 103 # Other directories are fine, but should be private. 104 data/... 105 graphics/... 106 collection_of_tests/... 107 code_relating_to_subfeature/... 108 109Module name 110----------- 111Pigweed upstream modules are always named with a prefix ``pw_`` to enforce 112namespacing. Projects using Pigweed that wish to make their own modules can use 113whatever name they like, but we suggest picking a short prefix to namespace 114your product (e.g. for an Internet of Toast project, perhaps the prefix could 115be ``it_``). 116 117C++ module structure 118-------------------- 119 120C++ public headers 121~~~~~~~~~~~~~~~~~~ 122Located ``{pw_module_dir}/public/<module>``. These headers are the public 123interface for the module. 124 125**Public headers** should take the form: 126 127``{pw_module_dir}/public/<module>/*.h`` 128 129**Exposed private headers** should take the form: 130 131``{pw_module_dir}/public/<module>/internal/*.h`` 132 133Examples: 134 135.. code-block:: 136 137 pw_foo/... 138 public/pw_foo/foo.h 139 public/pw_foo/a_header.h 140 public/pw_foo/baz.h 141 142For headers that must be exposed due to C++ limitations (i.e. are included from 143the public interface, but are not intended for use), place the headers in a 144``internal`` subfolder under the public headers directory; as 145``{pw_module_dir}/public/<module>/internal/*.h``. For example: 146 147.. code-block:: 148 149 pw_foo/... 150 public/pw_foo/internal/secret.h 151 public/pw_foo/internal/business.h 152 153.. note:: 154 155 These headers must not override headers from other modules. For 156 that, there is the ``public_overrides/`` directory. 157 158C++ public override headers 159~~~~~~~~~~~~~~~~~~~~~~~~~~~ 160Located ``{pw_module_dir}/public_overrides/<module>``. In general, the Pigweed 161philosophy is to avoid having "things hiding under rocks", and having header 162files with the same name that can override each other is considered a rock 163where surprising things can hide. Additionally, a design goal of the Pigweed 164module structure is to make it so there is ideally exactly one obvious place 165to find a header based on an ``#include``. 166 167However, in some cases header overrides are necessary to enable flexibly 168combining modules. To make this as explicit as possible, headers which override 169other headers must go in 170 171``{pw_module_dir}/public_overrides/...``` 172 173For example, the ``pw_unit_test`` module provides a header override for 174``gtest/gtest.h``. The structure of the module is (omitting some files): 175 176.. code-block:: 177 178 pw_unit_test/... 179 180 light_public_overrides/pw_unit_test/framework_backend.h 181 googletest_public_overrides/pw_unit_test/framework_backend.h 182 183 public_overrides/gtest 184 public_overrides/gtest/gtest.h 185 186 public/pw_unit_test 187 public/pw_unit_test/simple_printing_event_handler.h 188 public/pw_unit_test/event_handler.h 189 190Note that the overrides are in a separate directory ``public_overrides``. 191 192C++ implementation files 193~~~~~~~~~~~~~~~~~~~~~~~~ 194Located ``{pw_module_dir}/``. C++ implementation files go at the top level of 195the module. Implementation files must always use "" style includes. 196 197Example: 198 199.. code-block:: 200 201 pw_unit_test/... 202 main.cc 203 framework.cc 204 test.gni 205 BUILD.gn 206 README.md 207 208.. _module-structure-compile-time-configuration: 209 210Compile-time configuration 211~~~~~~~~~~~~~~~~~~~~~~~~~~ 212Pigweed modules are intended to be used in a wide variety of environments. 213In support of this, some modules expose compile-time configuration options. 214Pigweed has an established pattern for declaring and overriding module 215configuration. 216 217.. tip:: 218 219 Compile-time configuration provides flexibility, but also imposes 220 restrictions. A module can only have one configuration in a given build. 221 This makes testing modules with compile-time configuration more difficult. 222 Where appropriate, consider alternatives such as C++ templates or runtime 223 configuration. 224 225Declaring configuration 226^^^^^^^^^^^^^^^^^^^^^^^ 227Configuration options are declared in a header file as macros. If the macro is 228not already defined, a default definition is provided. Otherwise, nothing is 229done. Configuration headers may include ``static_assert`` statements to validate 230configuration values. 231 232.. code-block:: c++ 233 234 // Example configuration header 235 236 #ifndef PW_FOO_INPUT_BUFFER_SIZE_BYTES 237 #define PW_FOO_INPUT_BUFFER_SIZE_BYTES 128 238 #endif // PW_FOO_INPUT_BUFFER_SIZE_BYTES 239 240 static_assert(PW_FOO_INPUT_BUFFER_SIZE_BYTES >= 64); 241 242The configuration header may go in one of three places in the module, depending 243on whether the header should be exposed by the module or not. 244 245.. code-block:: 246 247 pw_foo/... 248 249 # Publicly accessible configuration header 250 public/pw_foo/config.h 251 252 # Internal configuration header that is included by other module headers 253 public/pw_foo/internal/config.h 254 255 # Internal configuration header 256 pw_foo_private/config.h 257 258The configuration header is provided by a build system library. This library 259acts as a :ref:`facade<docs-module-structure-facades>`. The details depend on 260the build system. 261 262GN compile-time configuration 263^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 264The facade uses a variable such as ``pw_foo_CONFIG``. In upstream Pigweed, all 265config facades default to the ``pw_build_DEFAULT_MODULE_CONFIG`` backend. The 266config facade is declared as follows: 267 268.. code-block:: 269 270 declare_args() { 271 # The build target that overrides the default configuration options for this 272 # module. This should point to a source set that provides defines through a 273 # public config (which may -include a file or add defines directly). 274 pw_foo_CONFIG = pw_build_DEFAULT_MODULE_CONFIG 275 } 276 277 # An example source set for each potential config header location follows. 278 279 # Publicly accessible configuration header (most common) 280 pw_source_set("config") { 281 public = [ "public/pw_foo/config.h" ] 282 public_configs = [ ":public_include_path" ] 283 public_deps = [ pw_foo_CONFIG ] 284 } 285 286 # Internal configuration header that is included by other module headers 287 pw_source_set("config") { 288 sources = [ "public/pw_foo/internal/config.h" ] 289 public_configs = [ ":public_include_path" ] 290 public_deps = [ pw_foo_CONFIG ] 291 visibility = [":*"] # Only allow this module to depend on ":config" 292 friend = [":*"] # Allow this module to access the config.h header. 293 } 294 295 # Internal configuration header 296 pw_source_set("config") { 297 public = [ "pw_foo_private/config.h" ] 298 public_deps = [ pw_foo_CONFIG ] 299 visibility = [":*"] # Only allow this module to depend on ":config" 300 } 301 302Bazel compile-time configuration 303^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 304The module that uses configuration depends on a ``label_flag``, conventionally 305named ``config``, that by default points to the 306``//pw_build:default_module_config``. For example, 307 308.. code-block:: python 309 310 # A module with a public config. That config doesn't need to be broken out 311 # into a separate cc_library. 312 cc_library( 313 name = "pw_foo", 314 hdrs = ["config.h"], 315 deps = [":config"], 316 ) 317 318 label_flag( 319 name = "config", 320 build_setting_default = "//pw_build:default_module_config", 321 ) 322 323 # A module with an internal config that's included by other module headers. 324 cc_library( 325 name = "pw_bar", 326 deps = [":internal_config"], 327 ) 328 329 cc_library( 330 name = "internal_config", 331 hdrs = ["config.h"], 332 deps = [":config"], 333 visibility = ["//visibility:private"], 334 ) 335 336 label_flag( 337 name = "config", 338 build_setting_default = "//pw_build:default_module_config", 339 ) 340 341 # A module with a private config. 342 cc_library( 343 name = "pw_bar", 344 implementation_deps = [":private_config"], 345 ) 346 347 cc_library( 348 name = "private_config", 349 hdrs = ["config.h"], 350 deps = [":config"], 351 visibility = ["//visibility:private"], 352 ) 353 354 label_flag( 355 name = "config", 356 build_setting_default = "//pw_build:default_module_config", 357 ) 358 359 360 361Overriding configuration 362^^^^^^^^^^^^^^^^^^^^^^^^ 363As noted above, all module configuration facades default to the same backend 364(``pw_build_DEFAULT_MODULE_CONFIG`` in GN, ``//pw_build:default_module_config`` 365in Bazel). This allows projects to override configuration values for multiple 366modules from a single configuration backend, if desired. The configuration 367values may also be overridden individually by setting backends for the 368individual module configurations (e.g. in GN, ``pw_foo_CONFIG = 369"//configuration:my_foo_config"``, in Bazel 370``--//pw_foo:config=//configuration:my_foo_config``). 371 372Configurations options are overridden by setting macros in the config backend. 373In Bazel, the only supported override mechanism are compilation options, such 374as ``-DPW_FOO_INPUT_BUFFER_SIZE_BYTES=256``. In GN and CMake, configuration 375macro definitions may also be set in a header file. The header file is included 376using the ``-include`` compilation option. 377 378GN config override example 379.......................... 380This example shows two ways to configure a module in the GN build system. 381 382.. code-block:: 383 384 # In the toolchain, set either pw_build_DEFAULT_MODULE_CONFIG or pw_foo_CONFIG 385 pw_build_DEFAULT_MODULE_CONFIG = get_path_info(":define_overrides", "abspath") 386 387 # This configuration sets PW_FOO_INPUT_BUFFER_SIZE_BYTES using the -D flag. 388 pw_source_set("define_overrides") { 389 public_configs = [ ":define_options" ] 390 } 391 392 config("define_options") { 393 defines = [ "PW_FOO_INPUT_BUFFER_SIZE_BYTES=256" ] 394 } 395 396 # This configuration sets PW_FOO_INPUT_BUFFER_SIZE_BYTES in a header file. 397 pw_source_set("include_overrides") { 398 public_configs = [ ":set_options_in_header_file" ] 399 400 # Header file with #define PW_FOO_INPUT_BUFFER_SIZE_BYTES 256 401 sources = [ "my_config_overrides.h" ] 402 } 403 404 config("set_options_in_header_file") { 405 cflags = [ 406 "-include", 407 rebase_path("my_config_overrides.h", root_build_dir), 408 ] 409 } 410 411Bazel config override example 412............................. 413This shows the only supported way to configure a module in Bazel. 414 415.. code-block:: python 416 417 # To use these overrides for all modules, set the global module config label 418 # flag, 419 # 420 # --@pigweed//pw_build:default_module_config=//path_to:config_overrides 421 # 422 # To use them just for one module, set the module-specific config label 423 # flag, 424 # 425 # --@pigweed//pw_foo:config_override=//path_to:config_overrides 426 cc_library( 427 name = "config_overrides", 428 defines = [ 429 "PW_FOO_INPUT_BUFFER_SIZE_BYTES=256", 430 ], 431 ) 432 433To conditionally enable targets based on whether a particular config override is 434enabled, a ``config_setting`` can be defined that looks at the config_override 435label flag value. This allows use of ``target_compatible_with`` to enable 436targets. 437 438.. code-block:: python 439 440 # Setup config_setting that is enabled when a particular config override is 441 # set. 442 config_setting( 443 name = "config_override_setting", 444 flag_values = { 445 "--@pigweed//pw_foo:config_override": ":config_overrides", 446 }, 447 ) 448 449 # For targets that need some specific config setting to build, conditionally 450 # enable them. 451 pw_cc_test( 452 name = "test", 453 target_compatible_with = select({ 454 ":config_override_setting": [], 455 "//conditions:default": ["@platforms//:incompatible"], 456 }), 457 ... 458 ) 459 460 461Why this config pattern is preferred 462^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 463Alternate patterns for configuring a module include overriding the module's 464config header or having that header optionally include a header at a known 465path (e.g. ``pw_foo/config_overrides.h``). There are a few downsides to these 466approaches: 467 468* The module needs its own config header that defines, provides defaults for, 469 and validates the configuration options. Replacing this header with a 470 user-defined header would require defining all options in the user's header, 471 which is cumbersome and brittle, and would bypass validation in the module's 472 header. 473* Including a config override header at a particular path would prevent 474 multiple modules from sharing the same configuration file. Multiple headers 475 could redirect to the same configuration file, but this would still require 476 creating a separate header and setting the config backend variable for each 477 module. 478* Optionally including a config override header requires boilerplate code that 479 would have to be duplicated in every configurable module. 480* An optional config override header file would silently be excluded if the 481 file path were accidentally misspelled. 482 483Python module structure 484----------------------- 485Python code is structured as described in the :ref:`docs-python-build-structure` 486section of :ref:`docs-python-build`. 487 488.. _docs-module-structure-facades: 489 490Facades 491------- 492In Pigweed, facades represent a dependency that can be swapped at compile time. 493Facades are similar in concept to a virtual interface, but the implementation is 494set by the build system. Runtime polymorphism with facades is not 495possible, and each facade may only have one implementation (backend) per 496toolchain compilation. 497 498In the simplest sense, a facade is just a dependency represented by a variable. 499For example, the ``pw_log`` facade is represented by the ``pw_log_BACKEND`` 500build variable. Facades typically are bundled with a build system library that 501depends on the backend. 502 503Facades are essential in some circumstances: 504 505* Low-level, platform-specific features (:ref:`module-pw_cpu_exception`). 506* Features that require a macro or non-virtual function interface 507 (:ref:`module-pw_log`, :ref:`module-pw_assert`). 508* Highly leveraged code where a virtual interface or callback is too costly or 509 cumbersome (:ref:`module-pw_tokenizer`). 510 511.. caution:: 512 513 Modules should only use facades when necessary. Facades are permanently locked 514 to a particular implementation at compile time. Multiple backends cannot be 515 used in one build, and runtime dependency injection is not possible, which 516 makes testing difficult. Where appropriate, modules should use other 517 mechanisms, such as virtual interfaces, callbacks, or templates, in place of 518 facades. 519 520The GN build system provides the 521:ref:`pw_facade template<module-pw_build-facade>` as a convenient way to declare 522facades. 523 524Multiple Facades 525~~~~~~~~~~~~~~~~ 526A module may contain multiple facades. Each facade's public override headers 527must be contained in separate folders in the backend implementation, so that 528it's possible to use multiple backends for a module. 529 530.. code-block:: 531 532 # pw_foo contains 2 facades, foo and bar 533 pw_foo/... 534 # Public headers 535 # public/pw_foo/foo.h #includes pw_foo_backend/foo.h 536 # public/pw_foo/bar.h #includes pw_foo_backend/bar.h 537 public/pw_foo/foo.h 538 public/pw_foo/bar.h 539 540 pw_foo_backend/... 541 542 # Public override headers for facade1 and facade2 go in separate folders 543 foo_public_overrides/pw_foo_backend/foo.h 544 bar_public_overrides/pw_foo_backend/bar.h 545 546Documentation 547------------- 548See :ref:`seed-0102`. 549 550Creating a new Pigweed module 551----------------------------- 552New Pigweed modules can be easily created using 553``pw module create MODULE_NAME`` (refer to the `Module name`_ guidelines). 554 555.. tip:: 556 557 Connect with the Pigweed community (by `mailing the Pigweed list 558 <https://groups.google.com/forum/#!forum/pigweed>`_ or `raising your idea 559 in the Pigweed chat <https://discord.gg/M9NSeTA>`_) to discuss your module 560 idea before getting too far into the implementation. This can prevent 561 accidentally duplicating work, or avoiding writing code that won't get 562 accepted. 563 564This command will create a folder with the provided module name as well as a 565number of basic build files. After this command has run, new code should be 566added in the following locations: 567 5681. Add `C++ public headers`_ files in 569 ``{pw_module_dir}/public/{pw_module_name}/`` 5702. Add `C++ implementation files`_ files in ``{pw_module_dir}/`` 5713. Add module documentation to ``{pw_module_dir}/docs.rst``. See 572 :ref:`seed-0102` for additional documentation options. 5734. Add GN build targets to ``{pw_module_dir}/BUILD.gn``. 574 575 - Add new ``pw_test`` targets to the list in ``pw_test_group("tests")``. 576 - Add any additional ``.rst`` documentation files to 577 ``pw_docs_group("docs")``. 578 5795. Add Bazel build targets to ``{pw_module_dir}/BUILD.bazel``. 580 5816. Add CMake build targets to ``{pw_module_dir}/CMakeLists.txt``. 582 5837. Run :ref:`module-pw_module-module-check` to ensure that the new module has 584 been properly configured. 585 586 - ``$ pw module check {pw_module_dir}`` 587 5888. Contribute your module to upstream Pigweed (optional but encouraged!) 589