count aggregation optimization inside one segment in log scenario [LUCENE-10425]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
In log scenario, we usually want to know the doc count of documents between every time intervals. One possible optimized method is to sort the docuemt in ascend order according to `@timestamp` field in one segment. then we can use this pr to find out the min/max docId in on time interval.
If there is no other filter query, the doc count of one time interval is (max docId- min docId +1)
if there is only one another term filter query, we can use this pr [https://github.com/apache/lucene/pull/688 ](https://github.com/apache/lucene/pull/688)to get the diff value of index, when we call advance(minId) and advance(maxId), the diff value is also the doc count of one time interval
---
Migrated from [LUCENE-10425](https://issues.apache.org/jira/browse/LUCENE-10425) by jianping weng (@wjp719), updated Mar 29 2022
Contributor guide
Research direction
Start by reading the log scenario context and the referenced Lucene pull requests #687 and #688, which describe doc ID range and iterator advancement behavior. The issue does not name files or tests and does not define a concrete acceptance criterion, so the implementation scope and definition of done need clarification.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100