kernelci / kernelci/dashboard

RFC: A/B comparison on tree details (builds, boots, tests)

Open
#1,951 10 comments 4 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
9
Forks
31
Avg merge
3d 19h
Merged PRs (30d)
16

Description

## Problem

Users compare two kernel commits by opening two tabs and eyeballing tables. The dashboard shows one revision at a time and period-over-period metrics, but not a deliberate **revision A vs revision B** view.

## Proposal

Users **pick the revisions they want to compare** (two commits on the same tree/branch). The UI may offer **suggestions** (previous commit, branch head, last visited, etc.) but does not auto-select sides.

We compare **pass/fail counts** (and inconclusive counts if shown) for **builds, boots, and tests** between the two revisions and surface **count deltas only** — no regression/fix labels for now, and not per-row status flips.

Same filters (`diffFilter`, origin, time window) apply to both sides. Comparison state is shareable via URL query params.

---

## Decided (or proposed to decide in this RFC)

| Topic | Proposal |
|-------|----------|
| **Comparable pair** | Two commits on the same `(tree, repo, branch)` |
| **What we compare** | **Counts** of passing and failing builds, boots, and tests (and inconclusive, if shown) — same buckets the dashboard already uses (`PASS`, `FAIL`, `groupStatus` inconclusive) |
| **Delta semantics** | `Δ pass = pass_B − pass_A`, `Δ fail = fail_B − pass_A` per entity and per group. Show deltas only; no regression/fix classification yet |
| **Grouping** | Tree-wide summary plus breakdown at an agreed grain (e.g. per config for builds, per path/platform for boots and tests) — exact keys TBD |
| **Filter parity** | Same filters on both sides before counts are computed |
| **Scope** | Tree Details: builds, boots, and tests |

### Example

```
Side A (abc1234) → Side B (def5678)

Builds: 400 pass · 12 fail → 395 pass · 18 fail (Δ -5 pass · +6 fail)
Boots: 380 pass · 8 fail → 372 pass · 14 fail (Δ -8 pass · +6 fail)
Tests: 1,200 pass · 50 fail → 1,180 pass · 72 fail (Δ -20 pass · +22 fail)

path: kselftest · x86_64 (tests)
A: 10 pass · 2 fail → B: 8 pass · 4 fail → Δ: -2 pass · +2 fail
```

---

## Rough UX

```
Side A: [pick revision] ↔ Side B: [pick revision]
suggestions: previous · branch head · …

[Builds | Boots | Tests] tab

Tree summary: pass_A · fail_A · pass_B · fail_B · Δ pass · Δ fail
Grouped table (grain TBD per tab)
```

---

## Success criteria

- Triage works without two tabs
- Comparison links are shareable and reproducible
- Acceptable latency on typical trees (target TBD after spike)

Contributor guide

Open the contributing guide

Research direction

Start at the Tree Details view and inspect the existing builds, boots, and tests tabs, including diffFilter, origin, and time-window handling. Trace how PASS, FAIL, and groupStatus counts are produced before defining shared revision selection and URL query state. Done means both selected revisions show reproducible count deltas with matching filters, though grouping and latency targets remain to be decided.

Written by the indexing model from the issue text.

Assessment

Domain
analytics, frontend, web-dev
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.