lance-format / lance-format/lance

Track Blob V2 Managed Kind

Open
#8,971 0 comments 0 reactions 1 assignee View on GitHub

@Xuanwo is already working on this.

Since Sep 3, 2026.

A-encoding A-format enhancement format-change
Dominant language
Rust
Stars
7.1k
Forks
852
Avg merge
3d 18h
Merged PRs (30d)
272

Description

Goal

Allow Blob v2 payload objects to survive DataFile rewrites without copying payload bytes by adding an independently addressed Managed kind.

Contract

  • Managed kind 4 uses the existing five descriptor fields in file formats 2.2 and 2.3, carrying an explicit base ID, relative URI, and byte range.
  • New objects use _blobs/<uuid>.blob. Existing Packed and Dedicated objects can be adopted at their original paths without copying bytes.
  • Compaction preserves object references. Cleanup respects protected snapshots; shallow clone retains source ownership and deep clone creates independently owned destination objects.
  • New Blob data-file commits atomically publish paired reader/writer capability bits and base bindings. The capability survives restore.
  • Existing Blob kinds retain their read semantics. All maintenance after activation must use a Managed-aware client, including maintenance through previously cached handles.

Delivery

  1. Document the existing Blob v2 contract: https://github.com/lance-format/lance/pull/9193
  2. Land a separate Managed-kind format proposal through the required format vote.
  3. Land the implementation: https://github.com/lance-format/lance/pull/9293
  4. Validate compaction, cleanup, clone, restore, and released-client compatibility.

Boundaries

No DataFile-attached Blob Reuse Index, configurable repack threshold, new repack API, or Blob v3 catalog is included.

Completion

The format contract and implementation are merged, lifecycle and compatibility coverage passes, and activation and maintenance requirements are documented.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.