1# S3 Benchmark Harness 2 3This module contains performance tests for `S3AsyncClient` and 4`S3TransferManager` 5 6## How to run 7 8``` 9# Build the JAR 10mvn clean install -pl :s3-benchmarks -P quick --am 11 12# download 13java -jar s3-benchmarks.jar --bucket=bucket --key=key -file=/path/to/destionfile/ --operation=download --partSizeInMB=20 --maxThroughput=100.0 14 15# upload 16java -jar s3-benchmarks.jar --bucket=bucket --key=key -file=/path/to/sourcefile/ --operation=upload --partSizeInMB=20 --maxThroughput=100.0 17``` 18 19## How to run S3Express (temporary) 20This benchmark is currently separate. Until it's integrated with the other benchmark runner, the usage is as follows: 21``` 22mvn exec:exec -Dexec.executable="java" -Dexec.args="-cp %classpath software.amazon.awssdk.s3benchmarks.s3express.S3BenchmarkRunner" 23``` 24 25## Command line arguments 26 27### Benchmark version 28 29The `--version` command line option is used to determine which component is under test: 30 31- `--version=crt` : Indicate to run the benchmark for the CRT's S3Client 32- `--version=java` : Indicate to run the benchmark for the java based S3 Async Client (`MultipartS3AsyncClient` class) 33- `--version=v2`: SDK v2 transfer manager (using `S3CrtAsyncClient` to delegate requests) 34- `--version=v1`: SDK v1 transfer manager (using `AmazonS3Client` to delegate requests) 35 36### Operation 37 38The `--operation` command line argument determine which transfer operation is used 39 40|operation|supported version| 41|---|-------| 42|download | v1 v2 java crt | 43|upload | v1 v2 java crt | 44|download_directory | v1 v2 | 45|upload_directory | v1 v2 | 46|copy | v1 v2 java | 47 48> All command line argument can be found in the `BenchmarkRunner` class. 49 50# Benchmark scripts Automation 51From the `.script` folder, use one of the `benchamrk` scripts to run a test suite. 52 53## single file benchmark 54### usage 55``` 56benchmark download|upload fs|tmpfs|no-op [<size>] 57``` 58The scripts require the operation 59(download or upload) and the location of the type of file system. 60- operation: `download|upload` 61- file system: `fs|tmpfs` 62 - `fs`: regular file system, under the root (`/`) directory 63 - `tmpfs`: in-memory file system (under the `/dev/shm` directory) 64- size (opt): if specified, will only run the test with the specified size (file must exist) 65 66The Benchmark suite will run the specified operation with different file sizes (1b 8MB+1 8MB-1 128MB 4GB 30GB, if no 67size are specified) and with 68the different client (v1 TM, v2 TM and CRT S3 client) and save the result for each benchmark under 69`result/$operation_$location_$name_$version_$size".txt` ie: `result/download_tmpfs_TMv2_128MB.txt`. 70For upload benchjmarks, the files 71`1b 8MB+1 8MB-1 128MB 4GB 30GB` must all exists under `/` for `fs` or `/dev/shm/` for `tmpfs`. The `create_benchmak_file` 72script can be used to create them. 73 74## copy benchmark 75``` 76benchmark-copy [<size>] 77``` 78Files `1b 8MB+1 8MB-1 128MB 4GB 30GB`, or the one passed as an argument, need to exist in the s3 bucket. 79 80## directory benchmark 81``` 82benchmark-dir download|upload fs|tmpfs [1B|4K|16M|5G] 83``` 84- `fs` is located ia `~/tm_dire_file` 85- `tmpfs` is located at `/dev/shm/tm_dir_file` 86 87# Graph scripts 88The `ploy.py` creates _Box and Whiskers_ type bar graphs of the test data. **The data is hard coped in the script file.** 89 90dependencies: [plotly](https://plotly.com/python/getting-started/) 91```bash 92pip install plotly 93``` 94 95creating static images also requires Kaleido ([more info](https://plotly.com/python/static-image-export/)) 96 97```bash 98pip install kaleido 99``` 100 101then simply run the `plot.py` script to generate images in `../images` (will be created if it does not exist) 102 103```bash 104python plot.py 105```