Stability Test Improvement Ideas

The following Stability Test improvements were discussed between Matt and Wesley on 4/18.

- Execution-id: Include the execution name in the execution id
- Execution-id date: ISO date should be readable in the execution name
- Have an option to run individual test suites and cases
- Report metrics have each day reporting from firelens datajet telling whether the test is alive
- Run the stability tests metric crunch periodically and compute the Stability Test metrics on failure rate per testcase. Add alarms when failure rate goes above a certain number.

The issue names Stability Tests, execution IDs, individual test suites and cases, Firelens Datajet metrics, metric crunching, and alarms, but no files or tests. Start by locating the Stability Test entry points and the code that generates execution IDs and computes metrics. Done requires a decided scope, implemented improvements, and verified reporting and failure-rate alarm behavior.

Written by the indexing model from the issue text.