lance-format / lance-format/lance
Track Blob V2 Managed Kind
Open
@Xuanwo is already working on this.
Since Sep 3, 2026.
A-encoding
A-format
enhancement
format-change
- Dominant language
- Rust
- Stars
- 7.1k
- Forks
- 852
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 272
Description
Goal
Allow Blob v2 payload objects to survive DataFile rewrites without copying payload bytes by adding an independently addressed Managed kind.
Contract
- Managed kind 4 uses the existing five descriptor fields in file formats 2.2 and 2.3, carrying an explicit base ID, relative URI, and byte range.
- New objects use
_blobs/<uuid>.blob. Existing Packed and Dedicated objects can be adopted at their original paths without copying bytes. - Compaction preserves object references. Cleanup respects protected snapshots; shallow clone retains source ownership and deep clone creates independently owned destination objects.
- New Blob data-file commits atomically publish paired reader/writer capability bits and base bindings. The capability survives restore.
- Existing Blob kinds retain their read semantics. All maintenance after activation must use a Managed-aware client, including maintenance through previously cached handles.
Delivery
- Document the existing Blob v2 contract: https://github.com/lance-format/lance/pull/9193
- Land a separate Managed-kind format proposal through the required format vote.
- Land the implementation: https://github.com/lance-format/lance/pull/9293
- Validate compaction, cleanup, clone, restore, and released-client compatibility.
Boundaries
No DataFile-attached Blob Reuse Index, configurable repack threshold, new repack API, or Blob v3 catalog is included.
Completion
The format contract and implementation are merged, lifecycle and compatibility coverage passes, and activation and maintenance requirements are documented.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.