kernelci / kernelci/dashboard

Provide failure, flaky test, and regression-focused summaries

Open
#2,097 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
9
Forks
31
Avg merge
3d 19h
Merged PRs (30d)
16

Description

Context

Maintainers running large test suites generally do not need to inspect successful results individually. User feedback highlighted workflows where thousands of tests run regularly, while the information requiring attention is primarily failures, flaky tests, errors, and newly introduced regressions.

Displaying large amounts of successful results makes it harder to identify actionable information.

Goal

Provide concise summaries focused on results that require developer attention. A maintainer should be able to quickly understand what failed, what appears flaky, and what changed compared with previous runs without navigating through large sets of successful results.

Points to discuss
• What should appear in the default summary.
• Distinction between failure, error, and flaky results.
• How flaky behavior should be detected and represented.
• Grouping repeated failures across environments.
• Integration with A/B Comparison.
• Whether summaries should be reusable by both the dashboard and reporting/notification mechanisms.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the issue's open questions about default summaries, failure/error/flaky distinctions, repeated failures across environments, and A/B Comparison. Review how the dashboard and reporting/notification mechanisms currently present test results; done means an agreed summary design that identifies failures, flaky tests, and regressions without requiring successful results to be inspected individually.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
analytics
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.