apache / apache/paimon

[Bug] Official version of the 0.5 paimon when writing in the primary key table updates' indexOutOfBoundsException 'error, try again using a snapshot of the version 0.6 when the error is still there

Open
#2,355 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
3.4k
Forks
1.4k
Avg merge
1d 11h
Merged PRs (30d)
396

Description

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/incubator-paimon/issues) and found nothing similar.

### Paimon version

0.5-incub and 0.6-11-20 snapshot version

### Compute Engine

FlinkSQL

### Minimal reproduce step

1th: kafka to paimon primary key and 1 bucket table , store as orc
2th:The length of a single field is close to 10 million
3th:The above error occurred while writing to the table continuously

### What doesn't meet your expectations?

Write to the primary key table normally, and update or add it normally

### Anything else?

No

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the continuous FlinkSQL write from Kafka into a one-bucket Paimon primary-key table stored as ORC, using a field near 10 million characters. Inspect the resulting IndexOutOfBoundsException and compare behavior across the reported Paimon versions. Done means primary-key inserts and updates complete normally without the error.

Written by the indexing model from the issue text.

Assessment

Tech stack
kafka, sql
Domain
data-engineering, databases, stream-processing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.