apache / apache/incubator-seata
Introduce automated flaky test tracking like OpenSearch
- Dominant language
- Java
- Stars
- 26k
- Forks
- 8.8k
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 4
Description
### Why you need it?
I would like to suggest introducing an automated flaky test tracking mechanism, similar to what [OpenSearch](https://github.com/opensearch-project/OpenSearch/issues/14308) has adopted.
### How it could be?
### Proposal
Introduce a similar system to automatically report flaky tests in Seata. This would help:
- Systematically track flaky behavior
- Prioritize test stability improvements
- Avoid merging regressions masked by inconsistent test results
### Implementation Ideas
- Hook into existing CI test result processing
- Identify tests that fail inconsistently across retries
- Auto-generate GitHub issues using a bot or GitHub Actions
- Apply consistent labels (e.g., flaky-test, test-failure, component:)
### Benefits
- Easier triage of test failures
- More transparent test health history
- Helps contributors focus on test reliability
Thanks 🙏
### Other related information
- https://learn.microsoft.com/en-us/azure/devops/report/powerbi/sample-test-analytics-flaky-tests?view=azure-devops&tabs=powerbi
Contributor guide
Assessment
This issue has not been assessed yet.