influxdata / influxdata/influxdb

Optimize general flux queries using equality on measurement names

Open
#22,485 0 comments 0 reactions 0 assignees View on GitHub
area/2.x area/flux area/performance
Dominant language
Rust
Stars
31.7k
Forks
3.7k
Avg merge
13h 37m
Merged PRs (30d)
8

Description

Flux queries with complex predicates involving measurement names that are not able to be statically evaluated by the optimization added in https://github.com/influxdata/influxdb/pull/22301 may take a long time to complete if the dataset includes a large amount of measurements. This is because in the `indexSet.measurementNamesByNameFilter` [method](https://github.com/influxdata/influxdb/blob/73043f94da37ef499c74621cd4c8908ea6906ad8/tsdb/index.go#L1466), the entire list of measurements in the index is iterated over, even when comparing for strict equality. It should be possible to short-circuit this logic by simply returning the name of the queried measurement if the comparison is being made on strict equality to speed these queries up.

Such queries are impossible to do with InfluxQL so this is lower priority, but may still be useful in the future as an easy optimization.

Contributor guide

Open the contributing guide

Research direction

Start in tsdb/index.go at indexSet.measurementNamesByNameFilter and review the optimization added in pull request 22301. Trace how strict equality predicates are handled, then confirm that equality can avoid iterating over every indexed measurement while preserving the existing behavior for other predicates.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.