IntersectMBO / IntersectMBO/lsm-tree
`lookups` should throw an error if the number of input keys is too large
- Dominant language
- Haskell
- Stars
- 53
- Forks
- 17
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 4
Description
> When calling `Database.LSMTree.lookups` with large-sized batches of keys, @jasagredo was seeing unexpected `NotFound` lookup results. The offending code is a typo in a bit-mask that we use while querying bloom filters. This typo effectively limited the number of bloom queries we could perform on a single run to `4096` (hexadecimal: `0xfff`), while the limit should have been `65536` (hexadecimal: `0xffff`). So under high `lookups` workloads, this limit would be hit and hence the erroneous lookup results.
Also: this limit should probably not be implicit. Instead, `lookups` should return an error if the number of input keys ever exceeds the limit of what we can handle in `lsm-tree`.
_Originally posted by @jorisdral in https://github.com/IntersectMBO/lsm-tree/issues/841#issuecomment-4308094157_
Contributor guide
Assessment
This issue has not been assessed yet.