lance-format / lance-format/lance

Update: action translation + conflict resolution

Open
#6,898 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Rust
Stars
7.1k
Forks
852
Avg merge
3d 18h
Merged PRs (30d)
272

Description

Parent PRD

Milestone: Action-based Transactions (UserOperation) — see milestone #11. Discussion: #5960. Design spike: #6448.

What to build

Operation::Update has no action translation and still uses a legacy build_manifest arm. The auxiliary-field design is settled in #6843; this issue is the implementation it defers to. Combined vertical slice — translation, apply routing, and conflict migration.

Translate Update per the #6843 decision:

  • RewriteRows: RemoveFragments(removed_fragment_ids) + AddFragments(new_fragments, preserved_row_ids) + UpdateDeletionVector per leftover partial delete + InvalidateIndexCoverage / RebindIndexCoverage over modified_field_ids + optional UpdateMergedGenerations.
  • RewriteColumns: one ReplaceFragmentColumns per (fragment, field-set) + InvalidateIndexCoverage.
  • update_mode is encoded structurally by which actions are emitted. inserted_rows_filter rides on AddFragments as conflict-detection metadata.

Then: route build_manifest through Action::apply and delete the legacy Update arm; implement reads() / writes() / rebase() for the actions involved; add a generator; port the legacy check_update_txn unit tests; extend the differential matrix; fill design-doc §5/§6.

This is the rebase-heaviest operation: rebase must handle affected_rows row-address intersection and inserted_rows_filter bloom-filter intersection on concurrent inserts. Round-trip losslessly.

conflict_resolver.rs must not be modified — it remains the production path and differential oracle until cutover (#6454).

Acceptance criteria

  • Update translates to actions for both RewriteRows and RewriteColumns; round-trip test passes
  • build_manifest routed through Action::apply; legacy Update arm deleted
  • reads/writes/rebase complete for the Update decomposition, including affected_rows and inserted_rows_filter handling in rebase
  • Property-based generator added
  • Legacy check_update_txn unit tests ported and green on the new resolver
  • Differential matrix extended; green for all migrated operations
  • Existing update / merge-insert tests still pass
  • conflict_resolver.rs unmodified

Blocked by

  • #6892 (conflict-resolution framework)
  • Depends on ReplaceFragmentColumns from #6842

User stories addressed

  • User story 5: single code path for applying operations
  • User story 6: single code path for conflict resolution
  • User story 9: old operations translated to actions

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with Operation::Update, build_manifest, and Action::apply, then read the decisions in #6843 and the dependencies #6892 and #6842. Use the legacy check_update_txn tests, the differential matrix, and the design document’s §5/§6 as guides; leave conflict_resolver.rs unchanged. Done means both update modes round-trip through actions, rebase handles affected_rows and inserted_rows_filter, and the listed tests and matrices pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
databases, distributed-systems
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.