Suggestion: Preserve prefix when normalizing keys
- Dominant language
- C
- Stars
- 33
- Forks
- 8
- Avg merge
- 8d 1h
- Merged PRs (30d)
- 1
Description
https://github.com/Kong/lua-resty-lmdb/commit/20ce979aa5797a55fd4638526129cf165e9941b7 implemented the function of automatically converting to sha256 when the key is larger than 511 bytes. But it also makes it impossible to use lmdb.prefix to retrieve these normalized keys.
In order to balance the prefix function and the normalization function while meeting the 511-byte limit, I suggest adding an option to control whether to retain part of the prefix.
Specific implementation:
If Preserve Prefix is turned off, keep it consistent with the status quo to ensure backward compatibility.
If Preserve Prefix is turned on:
1. MAX_KEY_SIZE is reduced to 511-64=447. Of course, another constant should be used in the specific implementation.
2. Calculate the sha256 of the complete KEY.
3. Extract the 477-byte prefix of the current KEY.
4. Concatenate the intercepted prefix and sha256 into a new key, as the normalized key.
Currently, I have implemented a layer of encapsulation when calling this library. If necessary, I am willing to submit a pull request to implement this function.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the normalization implementation introduced in commit 20ce979aa5797a55fd4638526129cf165e9941b7 and the lmdb.prefix retrieval path. Check the proposed prefix-preserving layout against the 511-byte limit, backward-compatible behavior when disabled, and retrieval of normalized keys; the open pull request is also relevant context.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- lua
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100