apache / apache/doris-flink-connector

[Bug] org.apache.doris.flink.exception.DorisRuntimeException: Index 2 out of bounds for length 2

Open
#571 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
385
Forks
279
Avg merge
2d 2h
Merged PRs (30d)
6

Description

### Search before asking

- [x] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues.

### Version

flink-doris-connector-1.20

flink 1.20

doris 3.0.4

### What's Wrong?

Caused by: org.apache.doris.flink.exception.DorisRuntimeException: Index 2 out of bounds for length 2
at org.apache.doris.flink.serialization.RowBatch.readArrow(RowBatch.java:195)
at org.apache.doris.flink.source.reader.DorisValueReader.hasNext(DorisValueReader.java:257)
at org.apache.doris.flink.source.reader.DorisSourceSplitReader.fetch(DorisSourceSplitReader.java:62)
at org.apache.flink.connector.base.source.reader.fetcher.FetchTask.run(FetchTask.java:58)
at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.runOnce(SplitFetcher.java:165)
... 6 more

测试代码如下(TEST_CASE):

// Table resultTable = tableEnv.sqlQuery("SELECT * FROM dim_parking_scd");
// right
// Table resultTable = tableEnv.sqlQuery("SELECT park_name FROM dim_parking_scd");
// right
// Table resultTable = tableEnv.sqlQuery("SELECT park_no FROM dim_parking_scd where start_date = '2025-03-03'");
// right
// Table resultTable = tableEnv.sqlQuery("SELECT park_no FROM dim_parking_scd where park_no = 'c区停车场'");
// 查询非主键字段
// right
// Table resultTable = tableEnv.sqlQuery("SELECT park_name FROM dim_parking_scd where park_no = 'c区停车场'");
// wrong(错误)
// Table resultTable = tableEnv.sqlQuery("SELECT park_name FROM dim_parking_scd where start_date = '2025-03-03'");
// wrong(错误)
// Table resultTable = tableEnv.sqlQuery("SELECT * FROM dim_parking_scd where start_date = '2025-03-03'");
// wrong(错误)
// Table resultTable = tableEnv.sqlQuery("SELECT park_no,start_date,park_name FROM dim_parking_scd where start_date = '2025-03-03'");

### What You Expected?

能用date类型doris主键字段做过滤条件,过滤出非主键字段的值

### How to Reproduce?

_No response_

### Anything Else?

_No response_

### Are you willing to submit PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)

Contributor guide

Open the contributing guide

Research direction

Start at RowBatch.java line 195, then trace how DorisValueReader.hasNext() handles the selected columns. Reproduce the reported queries that filter on the date primary-key field while selecting non-primary fields, and verify that the connector returns those values without the Index 2 out-of-bounds exception.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, sql
Domain
database
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.