1 //! Middleware that allows balancing load among multiple services.
2 //!
3 //! In larger systems, multiple endpoints are often available for a given service. As load
4 //! increases, you want to ensure that that load is spread evenly across the available services.
5 //! Otherwise, clients could see spikes in latency if their request goes to a particularly loaded
6 //! service, even when spare capacity is available to handle that request elsewhere.
7 //!
8 //! This module provides two pieces of middleware that helps with this type of load balancing:
9 //!
10 //! First, [`p2c`] implements the "[Power of Two Random Choices]" algorithm, a simple but robust
11 //! technique for spreading load across services with only inexact load measurements. Use this if
12 //! the set of available services is not within your control, and you simply want to spread load
13 //! among that set of services.
14 //!
15 //! [Power of Two Random Choices]: http://www.eecs.harvard.edu/~michaelm/postscripts/handbook2001.pdf
16 //!
17 //! Second, [`pool`] implements a dynamically sized pool of services. It estimates the overall
18 //! current load by tracking successful and unsuccessful calls to [`poll_ready`], and uses an
19 //! exponentially weighted moving average to add (using [`MakeService`]) or remove (by dropping)
20 //! services in response to increases or decreases in load. Use this if you are able to
21 //! dynamically add more service endpoints to the system to handle added load.
22 //!
23 //! # Examples
24 //!
25 //! ```rust
26 //! # #[cfg(feature = "util")]
27 //! # #[cfg(feature = "load")]
28 //! # fn warnings_are_errors() {
29 //! use tower::balance::p2c::Balance;
30 //! use tower::load::Load;
31 //! use tower::{Service, ServiceExt};
32 //! use futures_util::pin_mut;
33 //! # use futures_core::Stream;
34 //! # use futures_util::StreamExt;
35 //!
36 //! async fn spread<Req, S: Service<Req> + Load>(svc1: S, svc2: S, reqs: impl Stream<Item = Req>)
37 //! where
38 //!     S::Error: Into<tower::BoxError>,
39 //! # // this bound is pretty unfortunate, and the compiler does _not_ help
40 //!     S::Metric: std::fmt::Debug,
41 //! {
42 //!     // Spread load evenly across the two services
43 //!     let p2c = Balance::new(tower::discover::ServiceList::new(vec![svc1, svc2]));
44 //!
45 //!     // Issue all the requests that come in.
46 //!     // Some will go to svc1, some will go to svc2.
47 //!     pin_mut!(reqs);
48 //!     let mut responses = p2c.call_all(reqs);
49 //!     while let Some(rsp) = responses.next().await {
50 //!         // ...
51 //!     }
52 //! }
53 //! # }
54 //! ```
55 //!
56 //! [`MakeService`]: crate::MakeService
57 //! [`poll_ready`]: crate::Service::poll_ready
58 
59 pub mod error;
60 pub mod p2c;
61 pub mod pool;
62