influxdata / influxdata/influxdb

Queries to last cache with large `IN` list predicates seem slow

Open
#25,550 4 comments 0 reactions 0 assignees View on GitHub
kind/perf v3
Dominant language
Rust
Stars
31.7k
Forks
3.7k
Avg merge
13h 37m
Merged PRs (30d)
8

Description

The performance team has observed that when querying the last cache with `IN` clauses with ~100 elements leads to response times on the order of ~30ms.

For example,
```sql
SELECT * FROM last_cache('table_name', 'cache_name') where key_columns IN ('random value 1', ... 'random value 100');
```

This issue is for tracking whether or not this can be improved.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the SQL example against last_cache with an IN list of about 100 elements and measure the roughly 30 ms response time. Trace the last_cache query path and compare this case with smaller IN lists; done means demonstrating a measurable improvement or documenting why the behavior cannot be improved.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, sql
Domain
databases, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.