1.. _module-pw_docgen: 2 3========= 4pw_docgen 5========= 6.. pigweed-module:: 7 :name: pw_docgen 8 9``pw_docgen`` provides tools to generate documentation for Pigweed-based 10projects. 11 12.. note:: 13 14 Pigweed itself uses ``pw_docgen`` to generate ``pigweed.dev``. 15 16-------- 17Overview 18-------- 19Pigweed-based projects typically use a subset of Pigweed's modules and add their 20own product-specific modules on top of that, which may have product-specific 21documentation. ``pw_docgen`` provides a convenient way to combine all of the 22relevant documentation for a project into one place, allowing downstream 23consumers of release bundles (e.g. factory teams, QA teams, beta testers, etc.) 24to have a unified source of documentation early on. 25 26The documentation generation is integrated directly into the build system. Any 27build target can depend on documentation, which allows it to be included as part 28of a factory release build, for example. Additionally, documentation itself can 29depend on other build targets, such as :ref:`report cards <module-pw_bloat>` for 30binary size/profiling. Any time the code is changed, documentation will be 31regenerated with the updated reports. 32 33Each Pigweed module provides documentation describing its functionality, use 34cases, and programming API. 35 36Included in a module's documentation are report cards which show an overview of 37the module's size cost and performance benchmarks. These allow prospective users 38to evaluate the impact of including the module in their projects. 39 40----------------- 41Build integration 42----------------- 43Pigweed documentation files are written in `reStructuredText`_ format and 44rendered to HTML using `Sphinx`_ through Pigweed's GN build system. 45 46.. _reStructuredText: http://docutils.sourceforge.net/rst.html 47.. inclusive-language: ignore 48.. _Sphinx: http://www.sphinx-doc.org/en/master 49 50There are additonal Sphinx plugins used for rendering diagrams within 51reStructuredText files including: 52 53* `mermaid <https://mermaid-js.github.io/>`_ via the `sphinxcontrib-mermaid 54 <https://pypi.org/project/sphinxcontrib-mermaid/>`_ package. 55 56Documentation source and asset files are placed alongside code within a module 57and registered as a ``pw_doc_group`` target within a ``BUILD.gn`` file. These 58groups become available for import within a special documentation generation 59target, which accumulates all of them and renders the resulting HTML. This 60system can either be used directly within Pigweed, or integrated into a 61downstream project. 62 63GN templates 64============ 65 66pw_doc_group 67------------ 68The main template for defining documentation files is ``pw_doc_group``. It is 69used to logically group a collection of documentation source files and assets. 70Each Pigweed module is expected to provide at least one ``pw_doc_group`` target 71defining the module's documentation. A ``pw_doc_group`` can depend on other 72groups, causing them to be built with it. 73 74Arguments 75^^^^^^^^^ 76* ``sources``: RST documentation source files. 77* ``inputs``: Additional resources required for the docs (images, data files, 78 etc.) 79* ``group_deps``: Other ``pw_doc_group`` targets required by this one. 80* ``report_deps``: Report card generating targets (e.g. ``pw_size_diff``) on 81 which the docs depend. 82* ``other_deps``: Any other GN targets that should be run before this 83 ``pw_doc_group`` runs that is not included in one of the above ``dep`` 84 categories. 85 86Example 87^^^^^^^ 88.. code-block:: 89 90 pw_doc_group("my_doc_group") { 91 sources = [ "docs.rst" ] 92 inputs = [ "face-with-tears-of-joy-emoji.svg" ] 93 group_deps = [ ":sub_doc_group" ] 94 report_deps = [ ":my_size_report" ] 95 } 96 97pw_doc_gen 98---------- 99The ``pw_doc_gen`` template creates a target which renders complete HTML 100documentation for a project. It depends on registered ``pw_doc_group`` targets 101and creates an action which collects and renders them. 102 103To generate the complete docs, the template also requires a ``conf.py`` file 104configuring Sphinx's output, and a top level ``index.rst`` for the main page of 105the documentation. These are added at the root level of the built documentation 106to tie everything together. 107 108Arguments 109^^^^^^^^^ 110* ``conf``: Path to the ``conf.py`` to use for Sphinx. 111* ``index``: Path to the top-level ``index.rst`` file. 112* ``output_directory``: Directory in which to render HTML output. 113* ``deps``: List of all ``pw_doc_group`` targets required for the documentation. 114* ``python_metadata_deps``: Python-related dependencies that are only used as 115 deps for generating Python package metadata list, not the overall 116 documentation generation. This should rarely be used by non-Pigweed code. 117 118Example 119^^^^^^^ 120.. code-block:: 121 122 pw_doc_gen("my_docs") { 123 conf = "//my_docs/conf.py" 124 index = "//my_docs/index.rst" 125 output_directory = target_gen_dir 126 deps = [ 127 "//my_module:my_doc_group", 128 ] 129 } 130 131Generating documentation 132------------------------ 133All source files listed under a ``pw_doc_gen`` target and its ``pw_doc_group`` 134dependencies get copied out into a directory structure mirroring the original 135layout of the modules in which the sources appear. This is demonstrated below 136using a subset of Pigweed's core documentation. 137 138Consider the following target in ``$dir_pigweed/docs/BUILD.gn``: 139 140.. code-block:: 141 142 pw_doc_gen("docs") { 143 conf = "conf.py" 144 index = "index.rst" 145 output_directory = target_gen_dir 146 deps = [ 147 "$dir_pw_bloat:docs", 148 "$dir_pw_docgen:docs", 149 "$dir_pw_preprocessor:docs", 150 ] 151 } 152 153A documentation tree is created under the output directory. Each of the sources 154and inputs in the target's dependency graph is copied under this tree in the 155same directory structure as they appear under the root GN build directory 156(``$dir_pigweed`` in this case). The ``conf.py`` and ``index.rst`` provided 157directly to the ``pw_doc_gen`` template are copied in at the root of the tree. 158 159.. code-block:: 160 161 out/gen/docs/pw_docgen_tree/ 162 ├── conf.py 163 ├── index.rst 164 ├── pw_bloat 165 │ ├── bloat.rst 166 │ └── examples 167 │ └── simple_bloat.rst 168 ├── pw_docgen 169 │ └── docgen.rst 170 └── pw_preprocessor 171 └── docs.rst 172 173This is the documentation tree which gets passed to Sphinx to build HTML output. 174Imports within documentation files must be relative to this structure. In 175practice, relative imports from within modules' documentation groups are 176identical to the project's directory structure. The only special case is the 177top-level ``index.rst`` file's imports; they must start from the project's build 178root. 179 180Viewing documentation 181--------------------- 182``pw_docgen`` includes a web server that serves locally-generated documentation 183at ``pw_docgen.docserver``. It supports hot-reloading, so the rendered docs in 184your browser will refresh as you make changes to the source files. 185 186In most cases, you will not need to run the docs server directly. Instead, it 187will be run via :ref:`module-pw_watch`. 188 189----------------------------- 190pigweed.dev Sphinx extensions 191----------------------------- 192.. note:: The topics in this section only apply to ``pigweed.dev``. 193 This section isn't relevant to downstream users of ``pw_docgen``. 194 195This module houses Pigweed-specific extensions for the Sphinx documentation 196generator. Extensions are included and configured in ``docs/conf.py``. 197 198module_metadata 199=============== 200See :ref:`docs-contrib-docs-modules-metadata`. 201 202Canonical URL configuration 203--------------------------- 204``module_metadata`` fixes the canonical URLs for ``*/docs.html`` pages. By 205default Sphinx assumes that a page's canonical URL is its full URL. E.g. the 206default canonical URL for ``//pw_string/docs.rst`` is 207``https://pigweed.dev/pw_string/docs.html``. The ``pigweed.dev`` 208server treats ``https://pigweed.dev/pw_string/`` as the canonical URL however. 209This problem is not limited to module homepages; it occurs on any page that 210ends in ``/docs.html`` such as 211``https://pigweed.dev/third_party/emboss/docs.html``. ``module_metadata`` fixes 212this problem by ensuring that the ``<link rel="canonical" href="..."/>`` tag 213generated in the HTML is aligned with the server's configuration. 214 215After building the docs, the canonical URLs for all HTML pages can be verified 216by running the following command in a terminal from the root directory of the 217upstream Pigweed repo: 218 219.. code-block:: console 220 221 grep '<link rel="canonical' out/docs/gen/docs/html/* -R 222 223Context: :bug:`323077749`. 224 225bug 226--- 227This extension simplifies adding references to issues (bugs) in the Pigweed 228issue tracker. It defines a `Docutils role 229<https://docutils.sourceforge.io/docs/ref/rst/roles.html>`__ that can be 230used as follows. 231 232.. code-block:: rst 233 234 For more details see :bug:`323077749`. 235 236This becomes a hyperlink to https://pwbug.dev/323077749. 237 238google_analytics 239---------------- 240When this extension is included and a ``google_analytics_id`` is set in the 241Sphinx configuration, a Google Analytics tracking tag will be added to each 242page of the documentation when it is rendered to HTML. 243 244By default, the Sphinx configuration's ``google_analytics_id`` is set 245automatically based on the value of the GN argument 246``pw_docs_google_analytics_id``, allowing you to control whether tracking is 247enabled or not in your build configuration. Typically, you would only enable 248this for documentation builds intended for deployment on the web. 249 250Debugging Pigweed's Sphinx extensions 251------------------------------------- 252To step through your Pigweed extension code with 253`pdb <https://docs.python.org/3/library/pdb.html>`_: 254 255#. Set a breakpoint in your extension code: 256 257 .. code-block:: 258 259 breakpoint() 260 261#. Build ``python.install`` to install the code change into the bootstrap venv 262 (``environment/pigweed-venv/lib/python3.8/site-packages/pw_docgen``): 263 264 .. code-block:: 265 266 ninja -C out python.install 267 268#. Manually invoke Sphinx to build the docs and trigger your breakpoint: 269 270 .. code-block:: 271 272 cd out 273 sphinx-build -W -b html -d docs/gen/docs/help docs/gen/docs/pw_docgen_tree docs/gen/docs/html -v -v -v 274 275 You should see build output from Sphinx. The build should pause at your 276 breakpoint and you should then see pdb's prompt (``(Pdb)``). 277
