lance-format / lance-format/lance

Add a strict expected-head dataset commit mode

Open
#8,699 0 comments 0 reactions 1 assignee View on GitHub

@Xuanwo is already working on this.

Since Aug 21, 2026.

enhancement
Dominant language
Rust
Stars
7.1k
Forks
852
Avg merge
3d 18h
Merged PRs (30d)
272

Description

Goal

Allow a caller to commit against one exact dataset head and fail if that head has changed, without performing an initial conflict rebase.

Motivation

Some publication workflows validate a complete snapshot-dependent basis before staging output. The current commit path may rebase a non-overwrite transaction before its first commit attempt, so a concurrent change can land after final validation and still be accepted as compatible.

The limitation was surfaced while evaluating https://github.com/lancedb/lancedb/pull/3997.

Contract

  • The caller supplies an exact expected dataset version or equivalent head witness.
  • The first commit attempt fails with a typed conflict if the current head differs.
  • Strict mode performs no initial conflict rebase and no compatibility-based acceptance.
  • Retries cannot weaken or replace the original expected-head witness.
  • Default commit behavior remains unchanged for callers that do not request strict mode.
  • Object-store and commit-handler implementations expose equivalent semantics.

Completion gates

  • Local and object-store tests cover unchanged head, changed head, concurrent compatible changes, concurrent incompatible changes, and retries.
  • The strict path cannot publish after any intervening dataset commit.
  • Existing transaction conflict behavior remains unchanged outside strict mode.
  • Rust and required binding surfaces document the exact failure contract.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.