hadley / hadley/data-diff

Key retraction misfires on deletion-heavy diffs: weigh matched-row agreement

Open
#46 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
27
Forks
0
Avg merge
53m
Merged PRs (30d)
1

Description

Problem

When a guessed key is correct but the diff is dominated by row deletions, change_mass counts dropped rows in full, pushing changed mass past MAX_PLAUSIBLE_CHANGE_PERCENT (50%). The correct key is then retracted as excessive_change, and the second guess can land on a structurally-unique but semantically wrong column.

Reproducer from the SDVB benchmark (IMDB mixed_mixed_removal_rows2, 800 → 47 rows, a genuine 94% row deletion):

# declared key: honest answer
table_key([Series_Title], basis: declared)
row_drop(rows: 753)

# guessed key: retraction misfires
key_retracted([Series_Title], reason: excessive_change)
----
table_key([Overview], basis: guessed, overlap: 1.00)
table_regenerate()

The retracted key was producing the better diff; the re-guess on Overview (free text, unique on both sides) yields table_regenerate() — strictly less informative.

Proposed signal

The two cases the retraction trigger must distinguish leave different evidence in pass one:

  • Wrong key: matched rows disagree heavily in their cells.
  • Right key + massive deletion: matched rows are essentially clean; the change mass comes entirely from dropped rows.

In the example above, the 47 matched rows under Series_Title had ~zero changed cells, fully corroborating the key. A refinement: retract only when the matched rows themselves disagree heavily, or exempt deletion-dominated mass when matched rows are clean.

Found while running data-diff against the SDVB benchmark; also affects the WINE_small key_retracted cases (3 pairs).

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the SDVB IMDB mixed_mixed_removal_rows2 case and trace the pass-one key-retraction logic around change_mass and matched-row agreement. Done means deletion-dominated diffs with clean matched rows preserve the correct key, while keys whose matched rows disagree heavily can still be retracted; check the WINE_small cases as regression coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
50/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.