influxdata / influxdata/influxdb
Benchmark performance of slow path for tagKeys and tagValues
- Dominant language
- Rust
- Stars
- 31.7k
- Forks
- 3.7k
- Avg merge
- 13h 37m
- Merged PRs (30d)
- 8
Description
As part of these issues:
https://github.com/influxdata/influxdb/issues/19794
https://github.com/influxdata/influxdb/issues/19806
We added a "slow path" that does block scans in order to discover with field tag key/value combinations exist. For most cases, we can still use the index and shard metadata to answer metaqueries.
However, using the index and shard metadata will not provide answers that respect the time range passed into `tagKeys` and `tagValues`. It may be that we need to always perform the block scan in order to provide a correct answer. The work to do that is this issue:
https://github.com/influxdata/influxdb/issues/19814
But how slow is the slow path? Is it prohibitively slow to do it for every metaquery? We need a benchmark to help us answer this question.
So the work of this issue is:
- Create a database that is representative of a realistic scenario. Some research will be needed to determine what this means: how many series, how many fields, how many tag keys and values. Perhaps the product team could provide some input here.
- Measure performance of `tagKeys` and `tagValues` with an emphasis on how they tend to be invoked in the UI's Query Builder. To understand how the Query Builder works, I've found it useful to use a debugger to look at the metaqueries that are submitted when using the UI.
DOD:
- A reproducible benchmark exists and is merged to master
- We have some useful data on the performance of metaqueries
Contributor guide
Research direction
Start by reviewing issues 19794, 19806, and 19814, then inspect the existing slow path and use a debugger to observe the UI Query Builder's tagKeys and tagValues metaqueries. Create a realistic database, measure those queries, and add a reproducible benchmark. Done means the benchmark is merged to master and provides useful performance data.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- databases, performance, testing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100