1Go Protocol buffers 2=================== 3 4.. _proto_library: https://docs.bazel.build/versions/master/be/protocol-buffer.html#proto_library 5.. _default Go plugin: https://github.com/golang/protobuf 6.. _common plugins: #predefined-plugins 7.. _Go providers: /go/providers.rst 8.. _GoLibrary: /go/providers.rst#golibrary 9.. _GoSource: /go/providers.rst#gosource 10.. _GoArchive: /go/providers.rst#goarchive 11.. _Gazelle: https://github.com/bazelbuild/bazel-gazelle 12.. _Make variable substitution: https://docs.bazel.build/versions/master/be/make-variables.html#make-var-substitution 13.. _Bourne shell tokenization: https://docs.bazel.build/versions/master/be/common-definitions.html#sh-tokenization 14.. _gogoprotobuf: https://github.com/gogo/protobuf 15.. _compiler.bzl: compiler.bzl 16 17.. role:: param(kbd) 18.. role:: type(emphasis) 19.. role:: value(code) 20.. |mandatory| replace:: **mandatory value** 21 22rules_go provides rules that generate Go packages from .proto files. These 23packages can be imported like regular Go libraries. 24 25.. contents:: :depth: 2 26 27----- 28 29Overview 30-------- 31 32Protocol buffers are built with the three rules below. ``go_proto_library`` and 33``go_proto_compiler`` may be loaded from ``@io_bazel_rules_go//proto:def.bzl``. 34 35* `proto_library`_: This is a Bazel built-in rule. It lists a set of .proto 36 files in its ``srcs`` attribute and lists other ``proto_library`` dependencies 37 in its ``deps`` attribute. ``proto_library`` rules may be referenced by 38 language-specific code generation rules like ``java_proto_library`` and 39 ``go_proto_library``. 40* `go_proto_library`_: Generates Go code from .proto files using one or more 41 proto plugins, then builds that code into a Go library. ``go_proto_library`` 42 references ``proto_library`` sources via the ``proto`` attribute. They may 43 reference other ``go_proto_library`` and ``go_library`` dependencies via the 44 ``deps`` attributes. ``go_proto_library`` rules can be depended on or 45 embedded directly by ``go_library`` and ``go_binary``. 46* `go_proto_compiler`_: Defines a protoc plugin. By default, 47 ``go_proto_library`` generates Go code with the `default Go plugin`_, but 48 other plugins can be used by setting the ``compilers`` attribute. A few 49 `common plugins`_ are provided in ``@io_bazel_rules_go//proto``. 50 51The ``go_proto_compiler`` rule produces a `GoProtoCompiler`_ provider. If you 52need a greater degree of customization (for example, if you don't want to use 53protoc), you can implement a compatible rule that returns one of these. 54 55The ``go_proto_library`` rule produces the normal set of `Go providers`_. This 56makes it compatible with other Go rules for use in ``deps`` and ``embed`` 57attributes. 58 59Avoiding conflicts 60------------------ 61 62When linking programs that depend on protos, care must be taken to ensure that 63the same proto isn't registered by more than one package. This may happen if 64you depend on a ``go_proto_library`` and a vendored ``go_library`` generated 65from the same .proto files. You may see compile-time, link-time, or run-time 66errors as a result of this. 67 68There are two main ways to avoid conflicts. 69 70Option 1: Use go_proto_library exclusively 71~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 72 73You can avoid proto conflicts by using ``go_proto_library`` to generate code 74at build time and avoiding ``go_library`` rules based on pre-generated .pb.go 75files. 76 77Gazelle generates rules in this mode by default. When .proto files are present, 78it will generate ``go_proto_library`` rules and ``go_library`` rules that embed 79them (which are safe to use). Gazelle will automatically exclude .pb.go files 80that correspond to .proto files. If you have .proto files belonging to multiple 81packages in the same directory, add the following directives to your 82root build file: 83 84.. code:: bzl 85 86 # gazelle:proto package 87 # gazelle:proto_group go_package 88 89rules_go provides ``go_proto_library`` rules for commonly used proto libraries. 90The Well Known Types can be found in the ``@io_bazel_rules_go//proto/wkt`` 91package. There are implicit dependencies of ``go_proto_library`` rules 92that use the default compiler, so they don't need to be written 93explicitly in ``deps``. You can also find rules for Google APIs and gRPC in 94``@go_googleapis//``. You can list these rules with the commands: 95 96.. code:: bash 97 98 $ bazel query 'kind(go_proto_library, @io_bazel_rules_go//proto/wkt:all)' 99 $ bazel query 'kind(go_proto_library, @go_googleapis//...)' 100 101Some commonly used Go libraries, such as ``github.com/golang/protobuf/ptypes``, 102depend on the Well Known Types. In order to avoid conflicts when using these 103libraries, separate versions of these libraries are provided with 104``go_proto_library`` dependencies. Gazelle resolves imports of these libraries 105automatically. For example, it will resolve ``ptypes`` as 106``@com_github_golang_protobuf//ptypes:go_default_library_gen``. 107 108Option 2: Use pre-generated .pb.go files 109~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 110 111You can also avoid conflicts by generating .pb.go files ahead of time and using 112those exclusively instead of using ``go_proto_library``. This may be a better 113option for established Go projects that also need to build with ``go build``. 114 115Gazelle can generate rules for projects built in this mode. Add the following 116comment to your root build file: 117 118.. code:: bzl 119 120 # gazelle:proto disable_global 121 122This prevents Gazelle from generating ``go_proto_library`` rules. .pb.go files 123won't be excluded, and all special cases for imports (such as ``ptypes``) are 124disabled. 125 126If you have ``go_repository`` rules in your ``WORKSPACE`` file that may 127have protos, you'll also need to add 128``build_file_proto_mode = "disable_global"`` to those as well. 129 130.. code:: bzl 131 132 go_repository( 133 name = "com_example_some_project", 134 importpath = "example.com/some/project", 135 tag = "v0.1.2", 136 build_file_proto_mode = "disable_global", 137 ) 138 139A note on vendored .proto files 140~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 141 142By default, Bazel assumes imports in .proto files are relative to a repository 143root directory. This means, for example, if you import ``"foo/bar/baz.proto"``, 144that file must be in the directory ``foo/bar``, not 145``vendor/example.com/repo/foo/bar``. 146 147To deal with this, use the `strip_import_prefix` option in the proto_library_ 148for the vendored file. 149 150API 151--- 152 153go_proto_library 154~~~~~~~~~~~~~~~~ 155 156``go_proto_library`` generates a set of .go files from a set of .proto files 157(specified in a ``proto_library`` rule), then builds a Go library from those 158files. ``go_proto_library`` can be imported like any ``go_library`` rule. 159 160Providers 161^^^^^^^^^ 162 163* GoLibrary_ 164* GoSource_ 165* GoArchive_ 166 167Attributes 168^^^^^^^^^^ 169 170+---------------------+----------------------+-------------------------------------------------+ 171| **Name** | **Type** | **Default value** | 172+---------------------+----------------------+-------------------------------------------------+ 173| :param:`name` | :type:`string` | |mandatory| | 174+---------------------+----------------------+-------------------------------------------------+ 175| A unique name for this rule. | 176| | 177| By convention, and in order to interoperate cleanly with Gazelle_, this | 178| should be a name like ``foo_go_proto``, where ``foo`` is the Go package name | 179| or the last component of the proto package name (hopefully the same). The | 180| ``proto_library`` referenced by ``proto`` should be named ``foo_proto``. | 181+---------------------+----------------------+-------------------------------------------------+ 182| :param:`proto` | :type:`label` | |mandatory| | 183+---------------------+----------------------+-------------------------------------------------+ 184| Points to the ``proto_library`` containing the .proto sources this rule | 185| should generate code from. Avoid using this argument, use ``protos`` instead. | 186+---------------------+----------------------+-------------------------------------------------+ 187| :param:`protos` | :type:`label` | |mandatory| | 188+---------------------+----------------------+-------------------------------------------------+ 189| List of ``proto_library`` targets containing the .proto sources this rule should generate | 190| code from. This argument should be used instead of ``proto`` argument. | 191+---------------------+----------------------+-------------------------------------------------+ 192| :param:`deps` | :type:`label_list` | :value:`[]` | 193+---------------------+----------------------+-------------------------------------------------+ 194| List of Go libraries this library depends on directly. Usually, this will be | 195| a list of ``go_proto_library`` rules that correspond to the ``deps`` of the | 196| ``proto_library`` rule referenced by ``proto``. | 197| | 198| Additional dependencies may be added by the proto compiler. For example, the | 199| default compiler implicitly adds dependencies on the ``go_proto_library`` | 200| rules for the Well Known Types. | 201+---------------------+----------------------+-------------------------------------------------+ 202| :param:`importpath` | :type:`string` | |mandatory| | 203+---------------------+----------------------+-------------------------------------------------+ 204| The source import path of this library. Other libraries can import this | 205| library using this path. This must be specified in ``go_proto_library`` or | 206| inherited from one of the targets in ``embed``. | 207| | 208| ``importpath`` must match the import path specified in ``.proto`` files using | 209| ``option go_package``. The option determines how ``.pb.go`` files generated | 210| for protos importing this proto will import this package. | 211+---------------------+----------------------+-------------------------------------------------+ 212| :param:`importmap` | :type:`string` | :value:`""` | 213+---------------------+----------------------+-------------------------------------------------+ 214| The Go package path of this library. This is mostly only visible to the | 215| compiler and linker, but it may also be seen in stack traces. This may be | 216| set to prevent a binary from linking multiple packages with the same import | 217| path, e.g., from different vendor directories. | 218+---------------------+----------------------+-------------------------------------------------+ 219| :param:`embed` | :type:`label_list` | :value:`[]` | 220+---------------------+----------------------+-------------------------------------------------+ 221| List of Go libraries that should be combined with this library. The ``srcs`` | 222| and ``deps`` from these libraries will be incorporated into this library when it | 223| is compiled. Embedded libraries must have the same ``importpath`` and | 224| Go package name. | 225+---------------------+----------------------+-------------------------------------------------+ 226| :param:`gc_goopts` | :type:`string_list` | :value:`[]` | 227+---------------------+----------------------+-------------------------------------------------+ 228| List of flags to add to the Go compilation command when using the gc | 229| compiler. Subject to `Make variable substitution`_ and `Bourne shell tokenization`_. | 230+---------------------+----------------------+-------------------------------------------------+ 231| :param:`compiler` | :type:`label` | :value:`None` | 232+---------------------+----------------------+-------------------------------------------------+ 233| Equivalent to ``compilers`` with a single label. | 234+---------------------+----------------------+-------------------------------------------------+ 235| :param:`compilers` | :type:`label_list` | :value:`["@io_bazel_rules_go//proto:go_proto"]` | 236+---------------------+----------------------+-------------------------------------------------+ 237| List of rules producing `GoProtoCompiler`_ providers (normally | 238| `go_proto_compiler`_ rules). This is usually understood to be a list of | 239| protoc plugins used to generate Go code. See `Predefined plugins`_ for | 240| some options. | 241+---------------------+----------------------+-------------------------------------------------+ 242 243Example: Basic proto 244^^^^^^^^^^^^^^^^^^^^ 245 246Suppose you have two .proto files in separate packages: foo/foo.proto and 247bar/bar.proto. foo/foo.proto looks like this: 248 249.. code:: proto 250 251 syntax = "proto3"; 252 253 option go_package = "example.com/repo/foo"; 254 255 import "google/protobuf/any.proto"; 256 import "bar/bar.proto"; 257 258 message Foo { 259 bar.Bar x = 1; 260 google.protobuf.Any y = 2; 261 }; 262 263In foo/BUILD.bazel, we need to declare a ``proto_library`` rule that lists 264foo.proto in its ``srcs`` attribute. Since we import some other protos, we 265also need a label in ``deps`` for each imported package. We will need to 266create another ``proto_library`` in bar/BUILD.bazel, but we can use an 267existing library for any.proto, since it's one of the Well Known Types. 268 269.. code:: bzl 270 271 proto_library( 272 name = "foo_proto", 273 srcs = ["foo.proto"], 274 deps = [ 275 "//bar:bar_proto", 276 "@com_google_protobuf//:any_proto", 277 ], 278 visibility = ["//visibility:public"], 279 ) 280 281In order to these this proto in Go, we need to declare a ``go_proto_library`` 282that references to ``proto_library`` to be built via the ``proto`` attribute. 283Like ``go_library``, an ``importpath`` attribute needs to be declared. 284Ideally, this should match the ``option go_package`` declaration in the .proto 285file, but this is not required. We also need to list Go packages that the 286generated Go code imports in the ``deps`` attributes. Generally, ``deps`` 287in ``go_proto_library`` will correspond with ``deps`` in ``proto_library``, 288but the Well Known Types don't need to be listed (they are added automatically 289by the compiler in use). 290 291.. code:: bzl 292 293 load("@io_bazel_rules_go//proto:def.bzl", "go_proto_library") 294 295 go_proto_library( 296 name = "foo_go_proto", 297 importpath = "example.com/repo/foo", 298 proto = ":foo_proto", 299 visibility = ["//visibility:public"], 300 deps = ["//bar:bar_go_proto"], 301 ) 302 303This library can be imported like a regular Go library by other rules. 304 305.. code:: bzl 306 307 load("@io_bazel_rules_go//go:def.bzl", "go_binary") 308 309 go_binary( 310 name = "main", 311 srcs = ["main.go"], 312 deps = ["//foo:foo_go_proto"], 313 ) 314 315If you need to add additional source files to a package built from protos, 316you can do so with a separate ``go_library`` that embeds the 317``go_proto_library``. 318 319.. code:: bzl 320 321 load("@io_bazel_rules_go//go:def.bzl", "go_library") 322 323 go_library( 324 name = "foo", 325 srcs = ["extra.go"], 326 embed = [":foo_go_proto"], 327 importpath = "example.com/repo/foo", 328 visibility = ["//visibility:public"], 329 ) 330 331For convenience, ``proto_library``, ``go_proto_library``, and ``go_binary`` 332can all be generated by Gazelle_. 333 334Example: gRPC 335^^^^^^^^^^^^^ 336 337To compile protos that contain service definitions, just use the ``go_grpc`` 338plugin. 339 340.. code:: bzl 341 342 load("@io_bazel_rules_go//proto:def.bzl", "go_proto_library") 343 344 proto_library( 345 name = "foo_proto", 346 srcs = ["foo.proto"], 347 visibility = ["//visibility:public"], 348 ) 349 350 go_proto_library( 351 name = "foo_go_proto", 352 compilers = ["@io_bazel_rules_go//proto:go_grpc"], 353 importpath = "example.com/repo/foo", 354 proto = ":foo_proto", 355 visibility = ["//visibility:public"], 356 deps = ["//bar:bar_go_proto"], 357 ) 358 359go_proto_compiler 360~~~~~~~~~~~~~~~~~ 361 362``go_proto_compiler`` describes a plugin for protoc, the proto compiler. 363Different plugins will generate different Go code from the same protos. 364Compilers may be chosen through the ``compilers`` attribute of 365``go_proto_library``. 366 367Several instances of this rule are listed in `Predefined plugins`_. You will 368only need to use this rule directly if you need a plugin which is not there. 369 370Providers 371^^^^^^^^^ 372 373* GoProtoCompiler_ 374* GoLibrary_ 375* GoSource_ 376 377Attributes 378^^^^^^^^^^ 379 380+-----------------------------+----------------------+-----------------------------------------------------+ 381| **Name** | **Type** | **Default value** | 382+-----------------------------+----------------------+-----------------------------------------------------+ 383| :param:`name` | :type:`string` | |mandatory| | 384+-----------------------------+----------------------+-----------------------------------------------------+ 385| A unique name for this rule. | 386+-----------------------------+----------------------+-----------------------------------------------------+ 387| :param:`deps` | :type:`label_list` | :value:`[]` | 388+-----------------------------+----------------------+-----------------------------------------------------+ 389| List of Go libraries that Go code *generated by* this compiler depends on | 390| implicitly. Rules in this list must produce the `GoLibrary`_ provider. This | 391| should contain libraries for the Well Known Types at least. | 392+-----------------------------+----------------------+-----------------------------------------------------+ 393| :param:`options` | :type:`string_list` | :value:`[]` | 394+-----------------------------+----------------------+-----------------------------------------------------+ 395| List of command line options to be passed to the compiler. Each option will | 396| be preceded by ``--option``. | 397+-----------------------------+----------------------+-----------------------------------------------------+ 398| :param:`suffix` | :type:`string` | :value:`.pb.go` | 399+-----------------------------+----------------------+-----------------------------------------------------+ 400| File name suffix of generated Go files. ``go_proto_compiler`` assumes that | 401| one Go file will be generated for each input .proto file. Output file names | 402| will have the .proto suffix removed and this suffix appended. For example, | 403| ``foo.proto`` will become ``foo.pb.go``. | 404+-----------------------------+----------------------+-----------------------------------------------------+ 405| :param:`valid_archive` | :type:`bool` | :value:`True` | 406+-----------------------------+----------------------+-----------------------------------------------------+ 407| Whether code generated by this compiler can be compiled into a standalone | 408| archive file without additional sources. | 409+-----------------------------+----------------------+-----------------------------------------------------+ 410| :param:`import_path_option` | :type:`bool` | :value:`True` | 411+-----------------------------+----------------------+-----------------------------------------------------+ 412| When true, the ``importpath`` attribute from ``go_proto_library`` rules | 413| using this compiler will be passed to the compiler on the command line as | 414| ``--option import_path={}``. | 415+-----------------------------+----------------------+-----------------------------------------------------+ 416| :param:`plugin` | :type:`label` | :value:`@com_github_golang_protobuf//protoc-gen-go` | 417+-----------------------------+----------------------+-----------------------------------------------------+ 418| The plugin to use with protoc via the ``--plugin`` option. This rule must | 419| produce an executable file. | 420+-----------------------------+----------------------+-----------------------------------------------------+ 421 422Predefined plugins 423------------------ 424 425Several ``go_proto_compiler`` rules are predefined in 426``@io_bazel_rules_go//proto``. 427 428* ``go_proto``: default plugin from github.com/golang/protobuf. 429* ``go_grpc``: default gRPC plugin. 430* ``go_proto_validate``: validator plugin from 431 github.com/mwitkow/go-proto-validators. Generates ``Validate`` methods. 432* gogoprotobuf_ plugins for the variants ``combo``, ``gofast``, ``gogo``, 433 ``gogofast``, ``gogofaster``, ``gogoslick``, ``gogotypes``, ``gostring``. 434 For each variant, there is a regular version (e.g., ``gogo_proto``) and a 435 gRPC version (e.g., ``gogo_grpc``). 436 437Providers 438--------- 439 440Providers are objects produced by Bazel rules and consumed by other rules that 441depend on them. See `Go providers`_ for information about Go providers, 442specifically GoLibrary_, GoSource_, and GoArchive_. 443 444GoProtoCompiler 445~~~~~~~~~~~~~~~ 446 447GoProtoCompiler is the provider returned by the ``go_proto_compiler`` rule and 448anything compatible with it. The ``go_proto_library`` rule expects any rule 449listed in its ``compilers`` attribute to provide ``GoProtoCompiler``. If the 450``go_proto_compiler`` rule doesn't do what you need (e.g., you don't want to 451use protoc), you can write a new rule that produces this. 452 453``GoProtoCompiler`` is loaded from ``@io_bazel_rules_go//proto:def.bzl``. 454 455``GoProtoCompiler`` has the fields described below. Additional fields may be 456added to pass information to the ``compile`` function. This interface is 457*not final* and may change in the future. 458 459+-----------------------------+-------------------------------------------------+ 460| **Name** | **Type** | 461+-----------------------------+-------------------------------------------------+ 462| :param:`deps` | :type:`Target list` | 463+-----------------------------+-------------------------------------------------+ 464| A list of Go libraries to be added as dependencies to any | 465| ``go_proto_library`` compiled with this compiler. Each target must provide | 466| GoLibrary_, GoSource_, and GoArchive_. This list should include libraries | 467| for the Well Known Types and anything else considered "standard". | 468+-----------------------------+-------------------------------------------------+ 469| :param:`compile` | :type:`Function` | 470+-----------------------------+-------------------------------------------------+ 471| A function which declares output files and actions when called. See | 472| `compiler.bzl`_ for details. | 473+-----------------------------+-------------------------------------------------+ 474| :param:`valid_archive` | :type:`bool` | 475+-----------------------------+-------------------------------------------------+ 476| Whether the compiler produces a complete Go library. Compilers that just add | 477| methods to structs produced by other compilers will set this to false. | 478+-----------------------------+-------------------------------------------------+ 479 480Dependencies 481------------ 482 483In order to support protocol buffers, rules_go declares the external 484repositories listed below in ``go_rules_dependencies()``. These repositories 485will only be downloaded if proto rules are used. 486 487* ``@com_google_protobuf (github.com/google/protobuf)``: Well Known Types and 488 general proto support. 489* ``@com_github_golang_protobuf (github.com/golang/protobuf)``: standard 490 Go proto plugin. 491* ``@com_github_mwitkow_go_proto_validators 492 (github.com/mwitkow/go-proto-validators)``: validator plugin. 493* ``@com_github_gogo_protobuf (github.com/gogo/protobuf)``: gogoprotobuf 494 plugins. 495* ``@org_golang_google_grpc (github.com/grpc/grpc-go``: gRPC support. 496* gRPC dependencies 497 498 * ``@org_golang_x_net (golang.org/x/net)`` 499 * ``@org_golang_x_text (golang.org/x/text)`` 500 * ``@org_golang_google_genproto (google.golang.org/genproto)`` 501