demml / demml/scouter

Trace latency: add metrics summary fast path

Open
#289 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
13
Forks
1
PR merge metrics
No merged PRs in 30d

Description

Parent: #281

# Goal

Add a fast path for dashboard metrics and summary queries that should not require repeatedly scanning trace-span Parquet files.

Dashboards tend to issue repeated aggregate queries over recent windows. They should use precomputed or narrower summary data where correctness allows it.

# Scope

- Identify dashboard metrics and summary requests that can be served from summary tables or bounded aggregate paths.
- Keep raw trace-span scans available for requests that need span-level detail.
- Add freshness behavior that makes summary results safe under ongoing ingestion.
- Track fast-path hit rate, fallback rate, latency, and object-store request count.

# High-level design

The goal is not to move the whole analytical workload to Postgres. The goal is to avoid using the most expensive raw-span path for queries whose result shape is already summary-oriented.

This issue should clarify which queries are eligible for the fast path and which must stay on the raw DataFusion path.

# Acceptance criteria

- Eligible dashboard queries use the metrics/summary fast path.
- Ineligible queries fall back to the existing raw-span DataFusion path.
- Freshness behavior is explicit and tested.
- Metrics show fast-path hit rate and fallback reasons.
- Benchmarks show dashboard latency and object-store request-count impact.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.