lance-format / lance-format/lance
FSST follow-ups
Open
Nobody has claimed this yet.
A-encoding
enhancement
- Dominant language
- Rust
- Stars
- 7.1k
- Forks
- 852
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 272
Description
Some follow-up tasks from FSST.
- Allow FSST / dictionary to be toggled off using field metadata (useful for benchmarking / testing)
- Tune FSST minimum limit (does current 4MiB limit need to stay in place?)
- Figure out FSST packaging (should it be released as independent crate? Can we restructure packaging to match other crates?)
- Move symbol table from protobuf into separate buffer in page
- Make sure FSST is not applied on big endian systems (or enhance code to support big endian)
- #2606
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No files or tests are named. Start by selecting and scoping one unchecked FSST follow-up, then inspect the existing FSST implementation; done means the selected behavior is implemented, validated, and its checklist item can be closed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100