lance-format / lance-format/lance

Migrate metadata operations to action-level conflict resolution

Open
#6,893 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Rust
Stars
7.1k
Forks
852
Avg merge
3d 18h
Merged PRs (30d)
272

Description

Parent PRD

Milestone: Action-based Transactions (UserOperation) — see milestone #11. Discussion: #5960. Design spike: #6448.

What to build

UpdateConfig, UpdateBases, UpdateMemWalState, and ReserveFragments already translate to actions and route build_manifest through Action::apply (done under #6454). This issue migrates their conflict resolution onto the parallel action-level resolver built in #6892.

For the actions these operations use (UpdateConfig, AddBases, UpdateMergedGenerations, ReserveFragmentIds):

  • Implement reads() / writes() / rebase().
  • Add property-based transaction generators.
  • Port the legacy check_update_config_txn / check_add_bases_txn / check_update_mem_wal_state_txn / check_reserve_fragments_txn unit tests onto the new resolver.
  • Extend the differential matrix by these rows/columns; it must stay green for all operations migrated so far.
  • Fill the corresponding §5/§6 rules in the design doc.

Also completes UpdateConfig's translation to cover schema- and field-metadata updates: actions_from_operation currently returns None for these, so the schema/field-metadata conflicts that Operation::modifies_same_metadata detects would be silently lost. Split them onto the UpdateSchemaMetadata action per spike §4.

conflict_resolver.rs must not be modified — it remains the production path and differential oracle until cutover (#6454).

Acceptance criteria

  • reads/writes/rebase implemented for UpdateConfig, AddBases, UpdateMergedGenerations, ReserveFragmentIds
  • UpdateConfig translation covers schema/field-metadata (no longer returns None)
  • Property-based generators added
  • Legacy conflict unit tests ported and green on the new resolver
  • Differential matrix extended; green for all migrated operations
  • Design doc §5/§6 updated
  • conflict_resolver.rs unmodified

Blocked by

  • #6892 (conflict-resolution framework)

User stories addressed

  • User story 6: single code path for conflict resolution

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the conflict-resolution framework in #6892, the relevant action implementations, actions_from_operation, and the legacy check_update_config_txn, check_add_bases_txn, check_update_mem_wal_state_txn, and check_reserve_fragments_txn tests; keep conflict_resolver.rs unchanged. Compare the design spike and update the differential matrix and design doc §5/§6, with all migrated operations and property-based tests green when done.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
databases, distributed-systems
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.