apache / apache/doris-spark-connector

[Bug] There was a problem writing ARRAY<> into Doris

Open
#292 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
107
Forks
126
Avg merge
3d 10h
Merged PRs (30d)
4

Description

### Search before asking

- [x] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues.

### Version

spark-doris-connector-3.3_2.12-1.3.0.jar
spark = 3.3.1
doris = 2.1.7

### What's Wrong?

Reading ARRAY from Iceberg by Spark to Doris will be read as STRING type.

### What You Expected?

Reading ARRAY from Iceberg by Spark to Doris will be read as ARRAY type.

### How to Reproduce?

_No response_

### Anything Else?

_No response_

### Are you willing to submit PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)

Contributor guide

Open the contributing guide

Research direction

No file or test is named. Start by locating the connector's Spark-to-Doris type-mapping entry point and reproduce the reported Iceberg ARRAY case with Spark 3.3.1, Doris 2.1.7, and spark-doris-connector 1.3.0; done means the value is written as Doris ARRAY rather than STRING.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spark
Domain
data-engineering, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.