apache / apache/paimon

[Bug] Flipping deletion-vectors.bitmap64 on an existing table makes the next delete throw "Only instance with the same class type can be merged"

Open
#9,293 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
3.4k
Forks
1.4k
Avg merge
1d 11h
Merged PRs (30d)
396

Description

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.

### Paimon version

master, `fcae20748a777fa72763d7a0f046482cc9ceb6f8`

### Compute Engine

Java API for the reproduction below. The production call sites are Spark — `PaimonSparkWriter.scala:435` and `PaimonDeltaWriteBase.scala:206` — and Flink's `DataEvolutionDeleteOperator:241`.

### Minimal reproduce step

1. Create an append table with `deletion-vectors.enabled = true` and `deletion-vectors.bitmap64 = false` (the default).
2. Delete some rows, so a 32-bit deletion vector is written for a data file.
3. `ALTER TABLE ... SET ('deletion-vectors.bitmap64' = 'true')` — accepted, the option is not `@Immutable`.
4. Delete more rows from the same data file.

Step 4 fails:

```
java.lang.RuntimeException: Only instance with the same class type can be merged.
at org.apache.paimon.deletionvectors.Bitmap64DeletionVector.merge(Bitmap64DeletionVector.java:74)
at org.apache.paimon.deletionvectors.append.AppendDeleteFileMaintainer
.notifyNewDeletionVector(AppendDeleteFileMaintainer.java:134)
```

Reproduced at the maintainer level, which is the same call the Spark writer makes:

```java
// deletion vectors written with bitmap64 = false
TestAppendFileStore store = TestAppendFileStore.createAppendStore(tempDir, options);
CommitMessageImpl cm = store.writeDVIndexFiles(
BinaryRow.EMPTY_ROW, 0, Collections.singletonMap("f1", Arrays.asList(1, 3)));
store.commit(cm);
AppendDeleteFileMaintainer m = store.createDVIFMaintainer(BinaryRow.EMPTY_ROW, map);

DeletionVector.read(LocalFileIO.create(), map.get("f1")).getClass()
// -> BitmapDeletionVector

// what the writer produces once the option is flipped
DeletionVector fresh = new Bitmap64DeletionVector();
fresh.delete(7);
m.notifyNewDeletionVector("f1", fresh);
// -> RuntimeException: Only instance with the same class type can be merged.
```

And the option change in step 3 is accepted:

```
manager.commitChanges(SchemaChange.setOption("deletion-vectors.bitmap64", "true"))
// -> accepted, latest().options().get("deletion-vectors.bitmap64") == "true"
```

### What doesn't meet your expectations?

The two halves pick the implementation from different places and nothing reconciles them:

* `DeletionVector.read` (`DeletionVector.java:101-148`) dispatches on the **magic number in the file**, so a vector already on disk keeps its width no matter what the table option now says;
* new vectors are created from the **table option** — `BucketedDvMaintainer:51` and `DataEvolutionCompactDeletionVectorRewriter:231-233`;
* `merge` on both implementations rejects the other outright:

```java
// Bitmap64DeletionVector:70-76, and the mirror image in BitmapDeletionVector:58-64
if (deletionVector instanceof Bitmap64DeletionVector) {
roaringBitmap.or(((Bitmap64DeletionVector) deletionVector).roaringBitmap);
} else {
throw new RuntimeException("Only instance with the same class type can be merged.");
}
```

`deletion-vectors.bitmap64` carries no `@Immutable` annotation, so it is not in `CoreOptions.IMMUTABLE_OPTIONS` and `ALTER TABLE ... SET` lets it through. Since the option's documented purpose is Iceberg compatibility — *"only 64 bit bitmap implementation is compatible with Iceberg"* — turning it on for a table that already exists is exactly what someone would want to do.

Note that `BucketedDvMaintainer.notifyNewDeletion(fileName, position)` does not hit this: it goes through `computeIfAbsent`, so it keeps appending to whatever object was loaded from disk and quietly preserves the old width. Only the paths that build a fresh vector and then merge the previous one into it fail.

### Anything else?

Which way this should be resolved looks like a project decision rather than something to just patch, so I have not opened a PR:

1. **Mark the option `@Immutable`.** Smallest change, and it makes the failure a clear error at `ALTER` time instead of at the next delete. It also closes off migrating an existing table to the Iceberg-compatible format, which may be the main reason to touch the option at all.
2. **Let `merge` widen.** `Bitmap64DeletionVector.merge(BitmapDeletionVector)` can be total — every 32-bit position fits in 64 bits — so a flipped table would migrate file by file as deletes touch it. The reverse is not total: `BitmapDeletionVector.merge(Bitmap64DeletionVector)` has to reject positions above `Integer.MAX_VALUE`, so turning the option back off cannot be symmetric.
3. **Leave the behaviour and improve the message**, naming the option and the file so the cause is recoverable from the exception alone.

Happy to put up a PR for whichever of these you would prefer.

### Are you willing to submit a PR?

- [X] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the failure through AppendDeleteFileMaintainer.notifyNewDeletionVector, then read DeletionVector.java, BitmapDeletionVector.java, Bitmap64DeletionVector.java, and the option handling around BucketedDvMaintainer and DataEvolutionCompactDeletionVectorRewriter. Confirm how existing vectors are loaded after bitmap64 changes and discuss the three proposed resolutions with maintainers; done requires an agreed behavior and a regression test for the chosen path.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering, databases
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.