IntersectMBO / IntersectMBO/lsm-tree

`lookups` should throw an error if the number of input keys is too large

Open
#842 2 comments 0 reactions 0 assignees View on GitHub
bug lsm-tree priority: medium
Dominant language
Haskell
Stars
53
Forks
17
Avg merge
2d 3h
Merged PRs (30d)
4

Description

> When calling `Database.LSMTree.lookups` with large-sized batches of keys, @jasagredo was seeing unexpected `NotFound` lookup results. The offending code is a typo in a bit-mask that we use while querying bloom filters. This typo effectively limited the number of bloom queries we could perform on a single run to `4096` (hexadecimal: `0xfff`), while the limit should have been `65536` (hexadecimal: `0xffff`). So under high `lookups` workloads, this limit would be hit and hence the erroneous lookup results.

Also: this limit should probably not be implicit. Instead, `lookups` should return an error if the number of input keys ever exceeds the limit of what we can handle in `lsm-tree`.

_Originally posted by @jorisdral in https://github.com/IntersectMBO/lsm-tree/issues/841#issuecomment-4308094157_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.