ADORSYS-GIS / ADORSYS-GIS/lightbridge-code-intelligence

[Ticket]: Prompt Budget: coverage-loss panel shows No data instead of 0% when no run lost coverage

Đang mở
#600 0 bình luận 0 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
Rust
Star
0
Fork
0
Merge trung bình
14 giờ 13 phút
Pull request đã merge (30 ngày)
16

Mô tả

## Problem

On the Prompt Budget dashboard (added in #599), the **"Coverage loss — runs with omitted files"** panel displays **`No data`** when no run has lost coverage — i.e. in the healthy case.

Observed live on 2026-08-08 against production: 13 review runs on the `fast` preset, `Files omitted, total = 0`, and the coverage-loss panel rendering `No data`.

That is the *good* outcome being displayed in the same way a broken panel looks. On a dashboard whose entire purpose is a one-time sizing decision, "healthy" and "this query is broken" must not be visually identical — especially in a repo that has been repeatedly bitten by telemetry that silently returns nothing.

## Cause

The panel is a ratio:

```
100 * (sum by (fields_preset) (count_over_time( 0>)))
/ (sum by (fields_preset) (count_over_time()))
```

When no run has `diff_files_omitted > 0`, the numerator produces **no series at all** for that preset. A binary operation with a missing left-hand series yields no result — standard PromQL/LogQL vector-matching semantics, not a bug in itself. The denominator is fine; there is simply nothing to divide.

## Proposed fix

Substitute a zero-valued series carrying the denominator's labels when the numerator is absent — the usual PromQL idiom:

```
100 * (( or ( * 0)) / )
```

**Do not** solve this with Grafana's "No value → 0" panel option. That would be wrong: it cannot distinguish *"runs happened and none lost coverage"* (genuinely 0%) from *"no runs at all in this range"* (genuinely no data). Those are different states and an operator sizing a cap needs to tell them apart.

The query-level fix preserves that distinction correctly:

| state | numerator | denominator | result |
|---|---|---|---|
| runs happened, none lost coverage | absent | present | **0%** ✅ |
| runs happened, some lost coverage | present | present | real ratio ✅ |
| no runs at all in range | absent | absent | **No data** ✅ |

## Acceptance criteria

- [ ] Panel reads `0%` when diff-bearing runs exist and none have omitted files
- [ ] Panel still reads `No data` when there are no diff-bearing runs at all in the time range
- [ ] Per-preset breakdown is preserved (the fix must not collapse to a single unlabelled series — `or on() vector(0)` would do exactly that and is the wrong tool here)
- [ ] Change made in `tools/dashboard-gen/lci_dashboards/prompt_budget.py` and the JSON regenerated, never hand-edited

## Verification required — do not merge on a syntax read

LogQL's support for set operators between metric queries needs confirming for the exact expression used, and this has burned us twice already on this dashboard: #599 shipped four panels whose `count_over_time(...) by (...)` / `sum_over_time(...) by (...)` were outright parse errors, and an `avg_over_time` that silently returned 26666.67 instead of 40000. Both were found only by running the queries.

So: stand up a local Loki (`docker run grafana/loki`), push genuine `tracing`-emitted lines — real output from the actual code path, not hand-written JSON, since hand-writing the absent-field case inverts the result — and confirm all three rows of the table above with real query output. Take the query from the generated JSON, not the Python source.

`tools/dashboard-gen/lci_dashboards/prompt_budget.py`'s module docstring documents the method and the Loki versions already tested (3.1.1 and 3.5.7).

## Context

Follows #599. Not urgent — the panel is misleading, not wrong, and `Files omitted, total` already gives the same signal unambiguously. But it should be fixed before anyone uses this dashboard to actually pick a new `max_diff_chars`.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.