lance-format / lance-format/lance
Update `encoding.md` for `MiniBlockChunk` minichunk size changes
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 7.1k
- Forks
- 852
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 272
Description
PR #4959 expanded MiniBlockChunk minichunk size from u16 to u32, introducing new metadata and protobuf properties. The documentation at docs/src/format/file/encoding.md should be updated to reflect these changes:
- New protobuf property:
has_large_chunkforMiniBlockLayout. - New metadata key:
lance-encoding:minichunk-sizefor specifying mini chunk size (default 4KB, can be larger, e.g., 64KB). buffer_sizesnow stored asu32in memory:- Lance format v2.1: still stored as
u16on disk, converted tou32in memory. - Lance format v2.2: default mini chunk size remains 4KB, but users can override it (up to 64KB or larger) via the new metadata key.
- Lance format v2.1: still stored as
The docs should explain these changes so users and developers understand the updated MiniBlockChunk behavior.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read PR #4959 for the MiniBlockChunk changes, then update docs/src/format/file/encoding.md. Document the has_large_chunk protobuf property, the lance-encoding:minichunk-size metadata key, and the u32 buffer_sizes behavior across Lance format versions 2.1 and 2.2. Done means the file accurately explains the new metadata, defaults, overrides, and conversions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 1/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100