[Bug] Batch increment query has empty result when overwrite
- Dominant language
- Java
- Stars
- 3.4k
- Forks
- 1.4k
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 396
Description
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/incubator-paimon/issues) and found nothing similar.
### Paimon version
0.7
### Compute Engine
flink1.17
### Minimal reproduce step
when i batch insert overwrite paimon table , i can not use Batch increment query for any result.
```
CREATE TABLE if not EXISTS use_be_hours_2 (
user_id BIGINT,
item_id BIGINT,
behavior STRING,
dt STRING,
hh STRING,
PRIMARY KEY (user_id) NOT ENFORCED
)
with (
'manifest.format'='orc',
'changelog-producer' = 'input'
);
INSERT INTO use_be_hours_2
VALUES
(1, 1, 'watch', '2022-01-01', '10'),
(2, 2, 'like', '2022-01-01', '10');
INSERT OVERWRITE use_be_hours_2
VALUES
(1, 100, 'watch', '2022-01-01', '10'),
(3, 100, 'like', '2022-01-01', '10');
INSERT OVERWRITE use_be_hours_2
VALUES
(1, 200, 'watch', '2022-01-01', '10'),
(3, 200, 'like', '2022-01-01', '10'),
(4, 200, 'like', '2022-01-01', '10')
;
```
my query:
```
SELECT * FROM use_be_hours_2 /*+ OPTIONS('incremental-between' = '1,3') */;
SELECT * FROM use_be_hours_2$audit_log /*+ OPTIONS('incremental-between' = '1,3') */;
```
the result above both are empty , is it normal ?
### What doesn't meet your expectations?
the result above both are empty , is it normal ?
### Anything else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by running the provided Flink 1.17 SQL reproduction against Paimon 0.7, including both the table query and the audit_log query with incremental-between set to 1,3. Compare the results after the batch insert and two INSERT OVERWRITE operations; done means determining whether empty results are expected and, if not, identifying the affected incremental-query behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- sql
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100