lance-format / lance-format/lance

Improve CommitConflict error

Open
#3,223 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Rust
Stars
7.1k
Forks
852
Avg merge
3d 18h
Merged PRs (30d)
272

Description

CommitConflict error doesn't seem to be well understood. The problem is likely that with the transactions it is way too long. We should show a short Display of the transactions instead of a full Debug representation. If we shorten them, we can make the transactions in the message fields / attributes of the error, so they are still accessible if we need them to debug.

Example:

LanceDBClientError: Internal Server
Error: 500, error: {"code":500, "error": "LanceDB: database: N/A: lance error: Commit conflict for version 27: There was a concurrent commit that conflicts with t
his one and it cannot be automatically resolved. Please rerun the operation off the latest version of the table. \n Transaction: Transaction { read_version: 26, uuid: \"6269f1a3-3298-47f4-ad01-ef12
2eb07490\", operation: Delete { updated_fragments: [Fragment { id: 0, files: [DataFile { path: \"eb3579fa-Oaee-4ae6-b646-d08e0811fb27, lancel", fields: [0, 1, 2,
version: 0,
file_minor_version: 2 }1, deletion_file: Some(DeletionFile & read_version: 26, id: 13500629393622735842, file_type: Array,
Taun deeted rows: Some (12) 1), 0w 3d neta: None; phystcatr
ws: Some (26) }1,
deleted_fragment_ids: [1, predicate: \"path = 'docs/axolotl_readme.md'|" }, tag: None A\n Conflicting Transaction: Some(Transaction { read_version: 26, uuid: \"9de412bd-ef9e-496d-
a961-d05ddd4cb8f0\", operation: Delete { updated_fragments: [Fragment { id: 0, files: [DataFile { path: \"eb3579fa-Oaee-4a6-b646-d080811fb27. lancel", fields: [0, 1, 2, 31, column_indices: [1, f le_major_version: 0,
file_minor_version: 2 }l, deletion_file: Some(DeletionFile { read_version: 26,
id: 14627442727684223444,
file_type: Array, num_deleted_rows: Some (1) }), row_id_meta: None, phy
sical rows: Some(26) }], deleted fragment ids: [l, predicate: \"path = 'docs/test-again.md'|" }, tag: None }), /src/lance/rust/lance/src/io/commit.rs:111:23"}

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading src/lance/rust/lance/src/io/commit.rs around line 111 and trace how CommitConflict formats its transactions. Check nearby error-handling tests if present; done means the message uses a short transaction display while the transactions remain available as error fields or attributes.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.