influxdata / influxdata/influxdb
Add support for caching more complex predicates in TSI cache.
- Dominant language
- Rust
- Stars
- 31.7k
- Forks
- 3.7k
- Avg merge
- 13h 37m
- Merged PRs (30d)
- 8
Description
The TSI index has a cache that provides an in-memory reverse index from:
```
(measurement, tag_key, tag_value) ==> set_of_series_ids
```
So, given a measurement name (e.g., `cpu`), tag key (e.g., `region`) and tag value (e.g., `west`), the index can very quickly return a compressed bitset of all of the series ids matching that.
This makes TSI planning faster because all of the intersection, difference and union operations, when processing more complex predicates, are done on bitsets of integers.
However, in 2.0 we use measurement name for the organisation and bucket id, so that we can support multiple buckets within the same index. Typically then predicates are always going to require intersecting two bitsets, that satisfy:
```
"_measurement" = 'cpu' AND "region" = 'west'
```
We should consider extending the functionality of the cache to support tuples like this:
```
(measurement, _measurement, tag_key, tag_value) ==> set_of_series_ids
```
Contributor guide
Research direction
Start by locating the TSI index cache described in the issue and reviewing how it currently indexes (measurement, tag_key, tag_value) predicates. Done means the cache supports the proposed (measurement, _measurement, tag_key, tag_value) tuple and returns the corresponding series-id bitsets for predicates spanning multiple buckets.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100