Basekick-Labs / Basekick-Labs/arc
arcxrouter: buildEngineSQL lists the entire measurement — no time-based narrowing of the file list
- Dominant language
- Go
- Stars
- 677
- Forks
- 53
- Avg merge
- 9h 14m
- Merged PRs (30d)
- 164
Description
## What
`internal/arcxrouter/router.go#buildEngineSQL` builds the engine's file array by `h.Storage.List(ctx, db/measurement/)` over the **whole measurement** (all years), for every routed query. The arcx engine then does its own pruning — bind.rs has exactness-gated min/max statistics pruning and partition-path bucket math (partition.rs), which is data-time-exact — but only over the array it is handed, after the router has paid the full listing cost and materialized every path.
On a high-frequency workload this is the same live-hour cost class that query.file_time_pruning (#660) just fixed on the DuckDB path: a dashboard-shape query (`WHERE time > now() - INTERVAL ...`, the #650-#655 allow-list) routes to arcx and lists ~8K+ live-hour files plus all history, per query.
## Proposed
Reuse the pruner's range extraction when building the engine file list: narrow to hour paths from the query's time range (the same `ExtractTimeRange` the DuckDB path uses), and optionally apply the same filename flush-time pruning for the current hour. The engine's stats pruning stays as the exact second line of defense.
Context: arcx is in development, license-gated + private repo, so exposure is limited today — filed so the cost class is tracked before GA.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Contributor guide
Research direction
Start in internal/arcxrouter/router.go at buildEngineSQL and compare its h.Storage.List(ctx, db/measurement/) call with the DuckDB path's ExtractTimeRange usage. Read the referenced query.file_time_pruning work and the arcx pruning context in bind.rs and partition.rs. Done means routed arcx queries receive only time-relevant hour paths, with current-hour flush-time pruning where applicable, while engine stats pruning remains intact.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, sql
- Domain
- databases, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100