influxdata / influxdata/influxdb
Last 10 rows on big measurement
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 31.7k
- Forks
- 3.7k
- Avg merge
- 13h 37m
- Merged PRs (30d)
- 8
Description
I've an issue since the migration to influxdb 2. We have ameasurement with thousands of data by day and I need to know the last 10 rows in a month.
So previously, we have the following query : select * from payload where time > now() - 4w and time < now() order by desc limit 10
Now we translated it with :
from(bucket: "mybucket")
|> range(start: -4w, stop: now())
|> filter(fn: (r) => r._measurement == "payload")
|> pivot(rowKey:["_time"], columnKey: ["_field"], valueColumn: "_value")
|> group()
|> sort(columns: ["_time"], desc: true)
|> limit(n: 10)
But the time is very long !
It seems to be the "sort" function is very expensive.
Have you an idea to get my results faster ?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the Flux query over a large measurement, then examine the range, pivot, group, sort, and limit stages to identify where execution time is spent. Compare the query’s behavior with the reported legacy query and measure any candidate change; done means the last 10 rows for the requested period are returned with a demonstrated performance improvement.
Written by the indexing model from the issue text.
Assessment
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100