[Bug] Batch DELETE is a silent no-op on partial-update tables with remove-record-on-sequence-group
- Dominant language
- Java
- Stars
- 3.4k
- Forks
- 1.4k
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 396
Description
### Search before asking
- [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.
### Paimon version
master
### Compute Engine
Spark
### Minimal reproduce step
```sql
CREATE TABLE t (id INT, g INT, v BIGINT) TBLPROPERTIES (
'primary-key' = 'id',
'bucket' = '2',
'merge-engine' = 'partial-update',
'fields.g.sequence-group' = 'v',
'partial-update.remove-record-on-sequence-group' = 'g');
INSERT INTO t VALUES (1, 1, 10), (2, 1, 20);
DELETE FROM t WHERE id = 1;
SELECT * FROM t;
-- both rows are still there, the DELETE silently has no effect
```
### What doesn't meet your expectations?
`validatePKUpsertDeletable` allows batch DELETE when `partial-update.remove-record-on-sequence-group` is set, so the statement goes through the pk-upsert delete path (scan matching rows, mark them with `-D` and write back). The `-D` rows carry the real field values including the sequence-group field, so I'd expect `retractWithSequenceGroup` to set `currentDeleteRow` and remove the row — but nothing is deleted and no error is raised.
Not sure whether this is a bug or expected (e.g. the delete row is required to carry a *larger* sequence value than the current one?). If it is expected, maybe `validatePKUpsertDeletable` should reject this case or the behavior should be documented, since a silently no-op DELETE is quite surprising.
Noticed while working on #8837 (the fast path there is gated to DEDUPLICATE, so it's unrelated to that change — reproducible on vanilla master).
### Anything else?
_No response_
### Are you willing to submit a PR?
- [x] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by running the Spark SQL reproduction in the issue and inspect validatePKUpsertDeletable and retractWithSequenceGroup to trace how the sequence-group field is handled by batch DELETE. Compare the expected delete semantics with the observed -D rows, then make the behavior delete the matching rows or reject the unsupported case explicitly, with a regression test for the reproduction.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spark, sql
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100