influxdata / influxdata/influxdb

Use multi-measurement optimization for separate groups of OR'd measurements

Open
#22,491 0 comments 0 reactions 0 assignees View on GitHub
area/2.x area/flux area/performance
Dominant language
Rust
Stars
31.7k
Forks
3.7k
Avg merge
13h 37m
Merged PRs (30d)
8

Description

https://github.com/influxdata/influxdb/pull/22301 introduce an optimization for Flux queries containing multiple measurements, where the measurements are grouped together using multiple `OR` operators.

The initial implementation of this only allows for a single group of measurements. It should be possible to allow for multiple groups of measurements, provided they are all reachable via `AND` operators, and then taking the union of the measurement names.

This would be a fairly unusual type of query so it is probably not the highest priority. One fairly easy way to accomplish this would be to add the optimization check to the `measurementNamesByPredicate` [method](https://github.com/influxdata/influxdb/blob/c66a059101e12393c5e614661a17355519518785/tsdb/index.go#L1559) - this would allow the statically determined measurement names to be merged into the recursive tree appropriately if a suitable match was found in a subtree. This could also possibly boost performance of queries following the code path described in https://github.com/influxdata/influxdb/pull/22489.

We should also consider filtering the returned list from the optimization based on measurement names that actually exist - this should be fast to do using the index.

Contributor guide

Open the contributing guide

Research direction

Start in tsdb/index.go at measurementNamesByPredicate and review the optimization introduced by PR 22301, along with the query path discussed in PR 22489. Extend the matching logic to combine measurement groups reachable through AND operators and return their union; verify behavior with the relevant query or index tests found nearby.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases, performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.