infiniflow / infiniflow/infinity
[Feature Request]: United resource control for inverted index
- Dominant language
- C++
- Stars
- 4.7k
- Forks
- 445
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 7
Description
1. Using global thread pool or task scheduler for index building. Currently, a task executor is used at a columnar granularity, which is lack of being aware of global available threads.
2. Global memory quota control together with columnar store
3. Global segment id allocator
Contributor guide
Research direction
Start by mapping the inverted-index index-building path and its columnar-granularity task executor. Then examine how the columnar store manages memory and how segment IDs are allocated. Done means defining and implementing a coordinated approach for global thread scheduling, memory quotas, and segment ID allocation, with behavior verified across these paths.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- databases, search
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100