1# SDK Stability Regression Tests 2 3## Description 4This module contains stability regression tests. 5 6We define "stable" to mean that the SDK does not encounter any client-side errors for identified simple expected load 7scenarios, and the number of transient service or network I/O related errors are relatively small. 8 9## Test Case Acceptance 10 11As these tests will be running against live, running services, we can expect some subset of requests to fail due to transient 12failures, network disruptions, throttling, etc. We cannot expect that all requests will always succeed. Instead, we will establish 13that 5% or less of the total number of requests sent for a test case are allowed to fail. We specify that only errors that extend 14from SdkServiceException, or a form of network error such as IOException or ReadTimeoutException may be counted towards the 5%. 15Any other error type, such as SdkClientException will fail the test. 16 17 18## How to run 19 20- Run from your IDE 21 22- Run from maven command line 23 24``` 25mvn clean install -P stability-tests -pl :stability-tests 26``` 27 28- Build JAR and use the executable JAR 29 30First add tests to TestRunner Class, then run the following command. 31 32``` 33mvn clean install -pl :stability-tests --am -P quick 34mvn clean install -pl :bom-inernal 35cd test/stability-tests 36mvn package -P test-jar 37java -jar target/stability-tests-uber.jar 38``` 39 40## Adding New Tests 41 42- The tests are built using [JUnit 5](https://junit.org/junit5/). Make sure you are using the correct APIs and mixing of 43Junit 4 and Junit 5 APIs on the same test can have unexpected results. 44 45- All tests should have the suffix of `StabilityTests`, eg: `S3StabilityTests` 46 47 48 49