ByteBlockPool should allow different block sizes [LUCENE-2706]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
Currently ByteBlockPool has a fixed byteblock size of 1 << 15 bytes. This limit also makes BytesRefHash bound to 32kb buffers which is not ideal in many cases. For instance in MTQ (#4034) we might want to have a smaller block size of 1024 or 2048 for more efficiency. While this example suffers from another limitation since the blockSize used in TermsHashPerField is 32kb which in turn defines the size limit for a single term in the index. To make effective use of different byte block sizes we need to find a solution for this problem too but I think index statistics are on the way and max term length for a field could also make merges more efficient if we can alter the byte block size especially for fields with very short terms.
---
Migrated from [LUCENE-2706](https://issues.apache.org/jira/browse/LUCENE-2706) by Simon Willnauer (@s1monw)
Contributor guide
Assessment
This issue has not been assessed yet.