linkedin / linkedin/openhouse

Type 2 column-default rewrite gate should use commit deltas, not main-only

Open
#693 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Java
Stars
399
Forks
80
Avg merge
3d 23h
Merged PRs (30d)
15

Description

## Problem

`ReadBridgeStripProtection.isRewrite` treats a commit as a rewrite only if the **main-branch** snapshot (or a replace/RTAS flag) is overwrite/replace.

The snapshots PUT sends the table's **full** snapshot list and ref map. Main-only is how we avoid treating a historical overwrite still in that list as "this commit." Side effect: a WAP / named-branch overwrite leaves `main` on an old append, so Type 2 does not fire. An unaware client can overwrite a stamped table on a branch without sending `initial-default`.

## Do not

Do not infer the written ref by diffing `existing` vs `incoming` snapshot-ref maps. That re-derives what the commit already knew and fails on create-branch-at-same-snapshot, tags, and multi-ref PUTs.

## Do

[#669](https://github.com/linkedin/openhouse/pull/669) adds optional `jsonMetadataUpdates` on `IcebergSnapshotsRequestBody`: Iceberg REST `TableUpdate` deltas (`add-snapshot`, `set-snapshot-ref`) from `TableMetadata.changes()`.

After #669 (or equivalent) lands:

1. Plumb those deltas to the write path that calls `ReadBridgeStripProtection.prepare` (today they are audit-only).
2. Type 2: if an `add-snapshot` in this commit is overwrite/replace, require a matching `initial-default` handshake. Use `set-snapshot-ref` for which branch was written (`main`, WAP, …).
3. `CREATE BRANCH` with only `set-snapshot-ref` and no new snapshot is not a rewrite.
4. Clients that omit `jsonMetadataUpdates` keep the current main-only fallback.

Depends on #669. Blocks closing the WAP gap called out on #678.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with #669 and trace the write path that calls ReadBridgeStripProtection.prepare; inspect how jsonMetadataUpdates from Iceberg TableMetadata.changes() can reach it. Done means Type 2 uses add-snapshot and set-snapshot-ref for branch rewrites, ignores ref-only branch creation, and preserves the main-only fallback when deltas are omitted.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend-api-design, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.