apache / apache/paimon

[Bug] Query service process record repeatedly

Open
#4,379 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
3.4k
Forks
1.4k
Avg merge
1d 11h
Merged PRs (30d)
396

Description

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.

### Paimon version

Paimon 0.9

### Compute Engine

Flink 1.18

### Minimal reproduce step

1. Create paimon catelog
`
CREATE CATALOG paimon WITH (
'type'='paimon',
'warehouse'='file:/tmp/paimon'
);
`
`
USE CATALOG paimon;
`
2. Create table and insert one record
`CREATE TABLE DIM (k INT, v STRING, PRIMARY KEY(k) NOT ENFORCED) WITH ('bucket' = '1', 'deletion-vectors.enabled' = 'true');
`
`
INSERT INTO DIM VALUES (1, 'a');
`
3. Call Query Service
`
CALL sys.query_service('default.DIM', 1);
`
![image](https://github.com/user-attachments/assets/282d6098-4fa3-4be8-9d4e-a66cc1de5d6e)
![image](https://github.com/user-attachments/assets/2685aa8a-e3d4-4ef7-bcd0-2761e48dc32a)

![image](https://github.com/user-attachments/assets/7b9ead7a-67ac-43c9-bc86-149f621c0936)

4. Insert another record
`
INSERT INTO DIM VALUES (2, 'b');
`
It shows records sent 4
![image](https://github.com/user-attachments/assets/651158ec-9fa3-4172-b1c0-e73bc7f81de1)

### What doesn't meet your expectations?

Records sent should consistent with table record count

### Anything else?

_No response_

### Are you willing to submit a PR?

- [X] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the issue with Paimon 0.9 and Flink 1.18 using the CREATE CATALOG, table, INSERT, and CALL sys.query_service steps shown. Compare the records sent after the first and second inserts with the table record count. Done means the query service does not repeatedly process records and reports the expected count.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.