knative / knative/test-infra

Flaky test reporter reports severity of flakiness in Slack

Open
#1,374 5 comments 0 reactions 0 assignees View on GitHub
kind/enhancement kind/good-first-issue lifecycle/frozen
Dominant language
Go
Stars
83
Forks
160
PR merge metrics
No merged PRs in 30d

Description

Active flaky tests are reported as a list in Slack, provides a nice overview of which tests are flaky, but it's still not easy to determine how bad it is without clicking any link. It would be nice to show the severity of each test, for example failed 6/10 runs, or grade based on ratio such as low: 1-2, medium: 3-5, bad: 6-9, failed: 10

Contributor guide

Open the contributing guide

Research direction

Start by tracing the existing active flaky-test Slack reporter and how it represents failed and total runs. The issue names no files or tests, so repository search is required. Done means each listed test includes its failure count or an agreed severity grade in Slack.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
devops, testing
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.