1Load Balancing in gRPC 2====================== 3 4# Scope 5 6This document explains the design for load balancing within gRPC. 7 8# Background 9 10Load-balancing within gRPC happens on a per-call basis, not a 11per-connection basis. In other words, even if all requests come from a 12single client, we still want them to be load-balanced across all servers. 13 14# Architecture 15 16## Overview 17 18The gRPC client supports an API that allows load balancing policies to 19be implemented and plugged into gRPC. An LB policy is responsible for: 20- receiving updated configuration and list of server addresses from the 21 resolver 22- creating subchannels for the server addresses and managing their 23 connectivity behavior 24- setting the overall [connectivity state](connectivity-semantics-and-api.md) 25 (usually computed by aggregating the connectivity states of its subchannels) 26 of the channel 27- for each RPC sent on the channel, determining which subchannel to send 28 the RPC on 29 30There are a number of LB policies provided with gRPC. The most 31notable ones are `pick_first` (the default), `round_robin`, and 32`grpclb`. There are also a number of additional LB policies to support 33[xDS](grpc_xds_features.md), although they are not currently configurable 34directly. 35 36## Workflow 37 38Load-balancing policies fit into the gRPC client workflow in between 39name resolution and the connection to the server. Here's how it all 40works: 41 42 43 441. On startup, the gRPC client issues a [name resolution](naming.md) request 45 for the server name. The name will resolve to a list of IP addresses, 46 a [service config](service_config.md) that indicates which client-side 47 load-balancing policy to use (e.g., `round_robin` or `grpclb`) and 48 provides a configuration for that policy, and a set of attributes 49 (channel args in C-core). 502. The client instantiates the load balancing policy and passes it its 51 configuration from the service config, the list of IP addresses, and 52 the attributes. 533. The load balancing policy creates a set of subchannels for the IP 54 addresses of the servers (which might be different from the IP 55 addresses returned by the resolver; see below). It also watches the 56 subchannels' connectivity states and decides when each subchannel 57 should attempt to connect. 584. For each RPC sent, the load balancing policy decides which 59 subchannel (i.e., which server) the RPC should be sent to. 60 61See below for more information on `grpclb`. 62 63## Load Balancing Policies 64 65### `pick_first` 66 67This is the default LB policy if the service config does not specify any 68LB policy. It does not require any configuration. 69 70The `pick_first` policy takes a list of addresses from the resolver. It 71attempts to connect to those addresses one at a time, in order, until it 72finds one that is reachable. If none of the addresses are reachable, it 73sets the channel's state to TRANSIENT_FAILURE while it attempts to 74reconnect. Appropriate [backoff](connection-backoff.md) is applied for 75repeated connection attempts. 76 77If it is able to connect to one of the addresses, it sets the channel's 78state to READY, and then all RPCs sent on the channel will be sent to 79that address. If the connection to that address is later broken, 80the `pick_first` policy will put the channel into state IDLE, and it 81will not attempt to reconnect until the application requests that it 82does so (either via the channel's connectivity state API or by sending 83an RPC). 84 85### `round_robin` 86 87This LB policy is selected via the service config. It does not require 88any configuration. 89 90This policy takes a list of addresses from the resolver. It creates a 91subchannel for each of those addresses and constantly monitors the 92connectivity state of the subchannels. Whenever a subchannel becomes 93disconnected, the `round_robin` policy will ask it to reconnect, with 94appropriate connection [backoff](connection-backoff.md). 95 96The policy sets the channel's connectivity state by aggregating the 97states of the subchannels: 98- If any one subchannel is in READY state, the channel's state is READY. 99- Otherwise, if there is any subchannel in state CONNECTING, the channel's 100 state is CONNECTING. 101- Otherwise, if there is any subchannel in state IDLE, the channel's state is 102 IDLE. 103- Otherwise, if all subchannels are in state TRANSIENT_FAILURE, the channel's 104 state is TRANSIENT_FAILURE. 105 106Note that when a given subchannel reports TRANSIENT_FAILURE, it is 107considered to still be in TRANSIENT_FAILURE until it successfully 108reconnects and reports READY. In particular, we ignore the transition 109from TRANSIENT_FAILURE to CONNECTING. 110 111When an RPC is sent on the channel, the `round_robin` policy will 112iterate over all subchannels that are currently in READY state, sending 113each successive RPC to the next successive subchannel in the list, 114wrapping around to the start of the list when needed. 115 116### `grpclb` 117 118(This policy is deprecated. We recommend using [xDS](grpc_xds_features.md) 119instead.) 120 121This LB policy was originally intended as gRPC's primary extensibility 122mechanism for load balancing. The intent was that instead of adding new 123LB policies directly in the client, the client could implement only 124simple algorithms like `round_robin`, and any more complex algorithms 125would be provided by a look-aside load balancer. 126 127The client relies on the load balancer to provide _load balancing 128configuration_ and _the list of server addresses_ to which the client should 129send requests. The balancer updates the server list as needed to balance 130the load as well as handle server unavailability or health issues. The 131load balancer will make any necessary complex decisions and inform the 132client. The load balancer may communicate with the backend servers to 133collect load and health information. 134 135The `grpclb` policy uses the addresses returned by the resolver (if any) 136as fallback addresses, which are used when it loses contact with the 137balancers. 138 139The `grpclb` policy gets the list of addresses of the balancers to talk to 140via an attribute returned by the resolver. 141
