lance-format / lance-format/lance

Rewrite: RewriteFragments action + translation + conflict resolution

Open
#6,897 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Rust
Stars
7.1k
Forks
852
Avg merge
3d 18h
Merged PRs (30d)
272

Description

Parent PRD

Milestone: Action-based Transactions (UserOperation) — see milestone #11. Discussion: #5960. Design spike: #6448.

What to build

Operation::Rewrite has no action translation and still uses a legacy build_manifest arm. The RewriteFragments action it needs was specified by spike #6448 (§4 A5) but never added to the Action enum. This is a combined vertical slice — new action, translation, apply routing, and conflict migration.

  • Define the RewriteFragments action: payload { old_fragment_ids, new_fragments, preserves_row_ids }; implement apply and validate (asserts data equivalence so index coverage can be salvaged).
  • Translate Rewrite → one RewriteFragments per rewrite group, plus: with stable row IDs, RebindIndexCoverage + optional RegisterFragReuse; without stable row IDs, one RewriteIndex per rewritten index. Round-trip losslessly.
  • Route build_manifest through Action::apply and delete the legacy Rewrite arm.
  • Implement reads() / writes() / rebase() for RewriteFragments and any other action in the decomposition still lacking it.
  • Add a property-based generator; port the legacy check_rewrite_txn unit tests onto the new resolver; extend the differential matrix; fill design-doc §5/§6.

conflict_resolver.rs must not be modified — it remains the production path and differential oracle until cutover (#6454).

Acceptance criteria

  • RewriteFragments action defined; apply + validate implemented
  • Rewrite translates to actions; round-trip test passes
  • build_manifest routed through Action::apply; legacy Rewrite arm deleted
  • reads/writes/rebase complete for the Rewrite decomposition
  • Property-based generator added
  • Legacy check_rewrite_txn unit tests ported and green on the new resolver
  • Differential matrix extended; green for all migrated operations
  • Existing compaction tests still pass
  • conflict_resolver.rs unmodified

Blocked by

  • #6892 (conflict-resolution framework)

User stories addressed

  • User story 5: single code path for applying operations
  • User story 6: single code path for conflict resolution
  • User story 9: old operations translated to actions

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

This work is blocked by #6892 and spans the Rewrite action, build_manifest, Action::apply, the legacy check_rewrite_txn tests, the differential matrix, and design-doc §§5/6. Start by reading the Rewrite decomposition and spike #6448 §4 A5; do not modify conflict_resolver.rs. Done means the listed acceptance checks pass, including round-trip and property-based tests, while existing compaction tests remain green.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
data-engineering, databases, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.