Aggregate test results across hardware and configurations
- Dominant language
- Python
- Stars
- 9
- Forks
- 31
- Avg merge
- 3d 19h
- Merged PRs (30d)
- 16
Description
## Context
The dashboard currently tends to present individual test results as separate rows. When the same test runs on many hardware platforms or configurations, this can result in many rows representing essentially the same test.
Ted specifically described cases where the same test name appears many times with different hardware. During our own dashboard investigation, we also found cases where the same path appears multiple times with the same hardware and result status, while the attributes distinguishing those executions are not immediately visible.
Maintainers are often more interested in questions such as:
• Did this test fail everywhere or only on some systems?
• Does it fail only on arm64 but pass on x86_64?
• Does it correlate with a particular hardware characteristic?
• Does the same filesystem test fail across several filesystems or only one configuration?
## Goal
Explore ways to aggregate repeated results of the same test while preserving the dimensions necessary for investigation. Instead of requiring users to manually inspect many individual rows, the dashboard should make patterns across environments visible.
Points to discuss
• What makes two results equivalent enough to aggregate.
• Which attributes distinguish repeated executions that currently look identical in the table.
• Default grouping dimensions.
• How users expand an aggregate result into individual executions.
• Architecture and hardware grouping.
• Test configuration grouping.
• How failures, passes, and flaky results should be summarized.
• How this interacts with regression comparison views.
Contributor guide
Research direction
No files, tests, or entry points are named. Start by reviewing the dashboard's current individual-result rows and the dimensions available for hardware, configurations, paths, and statuses. Define grouping and expansion behavior, then validate that the design answers the listed cross-environment and regression-comparison questions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-visualization, frontend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100