lance-format / lance-format/lance

Default disable_transaction_file once v1.0.0+ readers are widespread

Open
#7,487 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Rust
Stars
7.1k
Forks
852
Avg merge
3d 18h
Merged PRs (30d)
272

Description

Background

The transaction is already embedded inline in the manifest (transaction_section) on every commit, so the separate _transactions/ file is redundant. FLAG_DISABLE_TRANSACTION_FILE (bit 32) lets a writer skip it and save one write per commit. Exposing a public toggle is tracked in #7486.

Inline-transaction writing and reader-side support for the flag shipped in v1.0.0 (2025-12-12). A dataset written with the external file disabled cannot be read by releases before v1.0.0 (they don't understand bit 32 and will reject it).

Ask

Once v1.0.0+ readers are sufficiently widespread (the usual ~3-6 month window after release, i.e. roughly mid-2026), flip the default so disable_transaction_file is on by default and we stop writing the redundant external file.

Before flipping:

  • Confirm read support for inline transaction_section has been stable across Rust, Python, and Java since v1.0.0.
  • Make sure the toggle from #7486 still allows opting back in to writing the external file for users who need older-reader compatibility.

See the Feature Maturity doc for the rationale and the downgrade-floor framing.

Blocked by #7486.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Feature Maturity document and the implementation and reader-support changes described for v1.0.0, then review the toggle planned in #7486. Confirm stable inline transaction support across Rust, Python, and Java and verify that opting back into the external file remains possible before changing the default; done means the compatibility checks pass and older-reader users retain an escape hatch.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, python, rust
Domain
data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.