lance-format / lance-format/lance
Action-level conflict resolution: framework + Append/Delete tracer bullet
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 7.1k
- Forks
- 852
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 272
Description
Parent PRD
Milestone: Action-based Transactions (UserOperation) — see milestone #11 for the full PRD.
Discussion: #5960. Conflict-resolution design spike: #6448.
Background
Spike #6448 produced two outputs: (a) the action catalog and (b) the action trait shape — a five-method contract (reads/writes/validate/rebase/apply) plus a driver loop. Output (a) was threaded into #6450–#6453; output (b) was never assigned to an implementation issue. As a result, build_manifest was migrated to Action::apply (9/13 ops, done under #6454), but conflict resolution still runs entirely through the legacy per-Operation check_*_txn matrix in rust/lance/src/io/commit/conflict_resolver.rs. The shipped Action::conflicts_with -> bool is the pre-spike shape and has no production callers.
Conflict resolution cannot be migrated one operation at a time in production: a conflict check is per pair of transactions, so during a transition a commit can hit a mixed (migrated, non-migrated) pair. The migration is therefore a parallel/shadow implementation, validated by a differential test matrix over the set of already-migrated operations (M×M), cutting over only once M = N.
This issue builds that framework and proves it end-to-end on Append + Delete.
What to build
- Land the spike design doc at
rust/lance-table/design/action_based_conflict_resolution.md(skeleton + #6448's resolved open questions; the §5/§6 conflict matrix is filled incrementally by the per-operation issues). - Implement the spike §7 trait on
Action:reads() -> ManifestMask,writes() -> ManifestMask,async rebase(&mut self, current, concurrent); generalizevalidateacross all actions. DefineManifestMask,TxnContext,ActionOutput. - Implement the transaction driver
commitloop (mask-gated rebase, spike §7). - Implement a parallel conflict-resolution entry point behind a legacy-shaped adapter: same
(other_transaction, other_version) -> Result<()>signature and rebased-transaction output asTransactionRebase::check_txn, so the eventual cutover is a one-line call-site swap. Not wired into the production commit path. - Implement
reads/writes/rebasefor the actions Append + Delete use:AddFragments,RemoveFragments,UpdateDeletionVector. Remaining actions get explicit unimplemented stubs, filled by their per-operation issue. - Handle
Restore/Cloneas top-level non-action transaction kinds in the driver. - Build the differential test harness + property-based transaction generator infrastructure; generators for Append and Delete.
- Port the legacy Append/Delete conflict unit tests onto the new resolver; feed the same fixtures through the differential harness so unit-test expectations are cross-checked against the legacy oracle.
conflict_resolver.rs must not be modified — it remains the production path and the differential oracle until the final cutover (#6454).
Acceptance criteria
- Design doc landed in-tree
-
reads/writes/validate/rebase/applytrait +ManifestMask/TxnContext/ActionOutputdefined - Driver
commitloop implemented - Parallel resolver entry point behind a legacy-shaped adapter
- Differential harness + property-based generators for Append/Delete
- M×M differential tests green for {Append, Delete}
- Legacy Append/Delete conflict unit tests ported and green on the new resolver
-
conflict_resolver.rsunmodified
Blocked by
None — startable immediately.
Blocks
Per-operation conflict-resolution migration, then the cutover:
- #6893 — metadata operations (UpdateConfig, UpdateBases, UpdateMemWalState, ReserveFragments)
- #6894 — schema operations (Project, Merge)
- #6895 — CreateIndex
- #6896 — Overwrite (translation + conflict)
- #6897 — Rewrite (translation + conflict)
- #6898 — Update (translation + conflict)
- #6842 — DataReplacement (translation + conflict)
- #6454 — final cutover
User stories addressed
- User story 6: single code path for conflict resolution
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with rust/lance-table/design/action_based_conflict_resolution.md and the §7 design from #6448, then inspect the existing transaction and action code around the conflict-resolution path. Build the parallel resolver without modifying rust/lance/src/io/commit/conflict_resolver.rs, and use the specified Append/Delete fixtures, generators, and differential tests as the completion check.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- databases, distributed-systems, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100