apache / apache/doris-spark-connector

Spark Doris Connector Release Note 25.0.0

Open
#268 0 comments 0 reactions 0 assignees View on GitHub
release-note
Dominant language
Java
Stars
107
Forks
126
Avg merge
3d 10h
Merged PRs (30d)
4

Description

# Spark Connector

## Feature

Support spark datasource V2 API pushdown. #250

Support partial limit pushdown for spark 3.2 and above. https://github.com/apache/doris-spark-connector/pull/257

## Improvement

Compatible with FE API changes. https://github.com/apache/doris-spark-connector/pull/260
Support create table statement for spark 3.3 and above. https://github.com/apache/doris-spark-connector/pull/264

## BugFix

Fix driver exception when spark catalog gets jdbc connection. https://github.com/apache/doris-spark-connector/pull/251

Fix data conversion exception when reading date type. https://github.com/apache/doris-spark-connector/pull/253

Fix the exception of loading arrow jdbc driver when loading mysql driver. https://github.com/apache/doris-spark-connector/pull/254

Fix the problem that the column order in the schema is inconsistent with the order in the data. https://github.com/apache/doris-spark-connector/pull/256

Fix exception when reading json type data via flight sql. https://github.com/apache/doris-spark-connector/pull/263

Fix the exception of reading date type caused by turning on `spark.sql.datetime.java8API.enabled` configuration. https://github.com/apache/doris-spark-connector/pull/265

# Spark Load

## Feature

Support job execution based on object storage. https://github.com/apache/doris-spark-connector/pull/266

# Thanks
@gnehil
@JNSimba

Contributor guide

Open the contributing guide

Research direction

Use the issue body as the release-note source and review the linked pull requests #250, #257, #260, #264, #251, #253, #254, #256, #263, #265, and #266. Confirm that the Spark Connector and Spark Load feature, improvement, and bug-fix entries are accurately represented, then ensure the 25.0.0 release note is complete.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spark
Domain
documentation, release
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.