[Bug] Flipping deletion-vectors.bitmap64 on an existing table makes the next delete throw "Only instance with the same class type can be merged"
- Dominant language
- Java
- Stars
- 3.4k
- Forks
- 1.4k
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 396
Description
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.
### Paimon version
master, `fcae20748a777fa72763d7a0f046482cc9ceb6f8`
### Compute Engine
Java API for the reproduction below. The production call sites are Spark — `PaimonSparkWriter.scala:435` and `PaimonDeltaWriteBase.scala:206` — and Flink's `DataEvolutionDeleteOperator:241`.
### Minimal reproduce step
1. Create an append table with `deletion-vectors.enabled = true` and `deletion-vectors.bitmap64 = false` (the default).
2. Delete some rows, so a 32-bit deletion vector is written for a data file.
3. `ALTER TABLE ... SET ('deletion-vectors.bitmap64' = 'true')` — accepted, the option is not `@Immutable`.
4. Delete more rows from the same data file.
Step 4 fails:
```
java.lang.RuntimeException: Only instance with the same class type can be merged.
at org.apache.paimon.deletionvectors.Bitmap64DeletionVector.merge(Bitmap64DeletionVector.java:74)
at org.apache.paimon.deletionvectors.append.AppendDeleteFileMaintainer
.notifyNewDeletionVector(AppendDeleteFileMaintainer.java:134)
```
Reproduced at the maintainer level, which is the same call the Spark writer makes:
```java
// deletion vectors written with bitmap64 = false
TestAppendFileStore store = TestAppendFileStore.createAppendStore(tempDir, options);
CommitMessageImpl cm = store.writeDVIndexFiles(
BinaryRow.EMPTY_ROW, 0, Collections.singletonMap("f1", Arrays.asList(1, 3)));
store.commit(cm);
AppendDeleteFileMaintainer m = store.createDVIFMaintainer(BinaryRow.EMPTY_ROW, map);
DeletionVector.read(LocalFileIO.create(), map.get("f1")).getClass()
// -> BitmapDeletionVector
// what the writer produces once the option is flipped
DeletionVector fresh = new Bitmap64DeletionVector();
fresh.delete(7);
m.notifyNewDeletionVector("f1", fresh);
// -> RuntimeException: Only instance with the same class type can be merged.
```
And the option change in step 3 is accepted:
```
manager.commitChanges(SchemaChange.setOption("deletion-vectors.bitmap64", "true"))
// -> accepted, latest().options().get("deletion-vectors.bitmap64") == "true"
```
### What doesn't meet your expectations?
The two halves pick the implementation from different places and nothing reconciles them:
* `DeletionVector.read` (`DeletionVector.java:101-148`) dispatches on the **magic number in the file**, so a vector already on disk keeps its width no matter what the table option now says;
* new vectors are created from the **table option** — `BucketedDvMaintainer:51` and `DataEvolutionCompactDeletionVectorRewriter:231-233`;
* `merge` on both implementations rejects the other outright:
```java
// Bitmap64DeletionVector:70-76, and the mirror image in BitmapDeletionVector:58-64
if (deletionVector instanceof Bitmap64DeletionVector) {
roaringBitmap.or(((Bitmap64DeletionVector) deletionVector).roaringBitmap);
} else {
throw new RuntimeException("Only instance with the same class type can be merged.");
}
```
`deletion-vectors.bitmap64` carries no `@Immutable` annotation, so it is not in `CoreOptions.IMMUTABLE_OPTIONS` and `ALTER TABLE ... SET` lets it through. Since the option's documented purpose is Iceberg compatibility — *"only 64 bit bitmap implementation is compatible with Iceberg"* — turning it on for a table that already exists is exactly what someone would want to do.
Note that `BucketedDvMaintainer.notifyNewDeletion(fileName, position)` does not hit this: it goes through `computeIfAbsent`, so it keeps appending to whatever object was loaded from disk and quietly preserves the old width. Only the paths that build a fresh vector and then merge the previous one into it fail.
### Anything else?
Which way this should be resolved looks like a project decision rather than something to just patch, so I have not opened a PR:
1. **Mark the option `@Immutable`.** Smallest change, and it makes the failure a clear error at `ALTER` time instead of at the next delete. It also closes off migrating an existing table to the Iceberg-compatible format, which may be the main reason to touch the option at all.
2. **Let `merge` widen.** `Bitmap64DeletionVector.merge(BitmapDeletionVector)` can be total — every 32-bit position fits in 64 bits — so a flipped table would migrate file by file as deletes touch it. The reverse is not total: `BitmapDeletionVector.merge(Bitmap64DeletionVector)` has to reject positions above `Integer.MAX_VALUE`, so turning the option back off cannot be symmetric.
3. **Leave the behaviour and improve the message**, naming the option and the file so the cause is recoverable from the exception alone.
Happy to put up a PR for whichever of these you would prefer.
### Are you willing to submit a PR?
- [X] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the failure through AppendDeleteFileMaintainer.notifyNewDeletionVector, then read DeletionVector.java, BitmapDeletionVector.java, Bitmap64DeletionVector.java, and the option handling around BucketedDvMaintainer and DataEvolutionCompactDeletionVectorRewriter. Confirm how existing vectors are loaded after bitmap64 changes and discuss the three proposed resolutions with maintainers; done requires an agreed behavior and a regression test for the chosen path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering, databases
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100