1## Criterion benchmarks for Tonic
2
3### Running the benchmarks
4From the root Tonic directory, `cargo bench`
5
6After running, the reports can be found in `tonic/target/criterion/report/index.html`
7
8[Gnuplot](http://www.gnuplot.info/) is required for graph generation.  If gnuplot is not installed, Criterion will display: `Gnuplot not found, disabling plotting` at the console.
9
10### Notes
111) Currently, these benchmarks only test the performance of constructing Tonic Requests and Responses, not over-the-wire throughput.
122) The `thrpt` value generated by Criterion is simply a measure of bytes consumed by the target function.
133) As we are not testing tonic-build compile time, the tests reference pre-compiled .rs files in 'benchmarks/compiled_protos'.
144) The original proto files are in the `proto` directory for reference.
155) This used the Criterion 3.0 `Criterion Group` functionality. Details here: https://docs.rs/criterion/0.3.0/criterion/
16
17### Interpreting Results
18
19Criterion is particularly useful for establishing a first-run baseline and then comparing after code-changes - e.g. `Performance has regressed` below.
20
21```bash
22Request_Response/request/100000
23                        time:   [2.7231 us 2.7588 us 2.7969 us]
24                        thrpt:  [33.298 GiB/s 33.758 GiB/s 34.200 GiB/s]
25                 change:
26                        time:   [+16.073% +17.871% +19.980%] (p = 0.00 < 0.05)
27                        thrpt:  [-16.653% -15.162% -13.847%]
28                        Performance has regressed.
29Found 3 outliers among 100 measurements (3.00%)
30  1 (1.00%) high mild
31  2 (2.00%) high severe
32```
33
34
35
36
37
38