cockroachdb / cockroachdb/cockroach

sql, test: automatic assignment of test failures when the test was recently modified

Open
#163,962 0 comments 0 reactions 0 assignees View on GitHub
A-testing C-enhancement T-sql-foundations
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

## Summary

A regular outcome from test failure triage is to find that the test itself was recently created or updated, and those recent changes introduced either flakiness or problems running in certain modes. Can we automate the process of detecting and reassigning these tests to the individual that made the recent change?

## Details

Some factors to consider:

- Was the test created or modified recently? How do the failure patterns align with the timing of that recent change? For example, a flaky test may not fail every time it is run, but we may have started to observe flakiness after a particular change was made.
- Has the issue already been detected? Have changes been made to the test since the last failure?
- Did the issue appear as soon as a new branch was cut? This assignment logic should not target individuals that may have touched test files in ways that do not impact their execution.
- Is the individual that authored the suspected change a Cockroach Labs employee? We do not want to automatically assign issues to external contributors.

Epic: CRDB-60540

Jira issue: CRDB-60550

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.