Estimate memory usage for merges
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
### Description
Merges should expose an estimation of the amount of heap memory that will be used for the actual merge to be done.
This information should be exposed as part of the merge information, so merge schedulers can check the amount of memory needed to perform the merge and thus make decisions on throttling merges that would take too much memory.
As #14208 describes, this is specially important for structures like HNSW graphs that can take a huge amount of memory during merges.
Contributor guide
Research direction
The issue names no files or tests. Start by locating the merge information exposed to merge schedulers and read the context from #14208, especially the HNSW merge case. Done means merge information includes a heap-memory estimate that schedulers can use when making throttling decisions, with tests covering the estimate.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, search
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100