[Bug] Contradictory partition-key equals are treated as a partition drop
- Dominant language
- Java
- Stars
- 3.4k
- Forks
- 1.4k
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 396
Description
### Search before asking
- [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.
### Paimon version
master, `475be566f` (2.1-SNAPSHOT).
### Compute Engine
Flink. `SupportsRowLevelOperationFlinkTableSink` is the only production user of this visitor.
### Minimal reproduce step
This one is a contract problem rather than something I could trigger from SQL, and I want to be upfront about that.
`OnlyPartitionKeyEqualVisitor` decides whether a DELETE can be executed by dropping partitions. It collects the equality literals into a map, one entry per partition key:
```java
public Boolean visitEqual(FieldRef fieldRef, Object literal) {
boolean contains = partitionKeys.contains(fieldRef.name());
if (contains) {
partitions.put(fieldRef.name(), literal.toString());
return true;
}
return false;
}
```
Two equality predicates on the same key therefore overwrite each other, and the conjunction still reports "droppable". Feeding it `pt = 'a' AND pt = 'b'`:
```java
OnlyPartitionKeyEqualVisitor visitor = new OnlyPartitionKeyEqualVisitor(Arrays.asList("pt", "dt"));
Predicate contradiction = PredicateBuilder.and(equal(pt, "a"), equal(pt, "b"));
contradiction.visit(visitor); // true
visitor.partitions(); // {pt=b}
```
The Flink sink reads that as a partition drop of `pt = 'b'` and deletes every row in that partition, while the predicate itself matches nothing.
What I could not do is get such a predicate to the sink from SQL. `DELETE FROM t WHERE pt = 'a' AND pt = 'b'` is folded to FALSE by Calcite's simplification before `applyDeleteFilters` runs, and `PredicateBuilder.and` does not merge equality predicates itself, so the visitor only sees two `Equal` leaves if something hands them over unfolded. So today the guarantee that this cannot happen lives in the planner, not in Paimon.
### What doesn't meet your expectations?
The visitor reduces a conjunction of constraints to a last-write-wins map, which loses the information that the constraints conflict. For a decision whose consequence is dropping a whole partition, that seems worth deciding inside Paimon rather than relying on an external optimizer to never pass a contradiction through. A future engine version, another engine binding, or a caller that builds the predicate directly would not have that protection.
### Anything else?
The same class already returns false for everything it does not understand (`visitIsNull`, `visitIn`, ranges, OR), so a conservative answer for a contradiction fits how it is written.
### Are you willing to submit a PR?
- [x] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with OnlyPartitionKeyEqualVisitor, especially visitEqual and partitions(), then inspect its use by SupportsRowLevelOperationFlinkTableSink. Trace how a conjunction is classified and add coverage for contradictory equalities such as pt = 'a' AND pt = 'b'. Done means the contradiction is not considered droppable and cannot select a partition for deletion.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 76/100