apache / apache/doris-spark-connector
[Feature] 优化 spark-doris-connector jar 大小
- Dominant language
- Java
- Stars
- 107
- Forks
- 126
- Avg merge
- 3d 10h
- Merged PRs (30d)
- 4
Description
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues.
### Description
建议优化一下spark-doris-connector jar 大小,当前jar 中包含了很多spark jars目录已经存在jar。
还包含calcite jar 是否必要?
### Use case
_No response_
### Related issues
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
Contributor guide
Research direction
Start by inspecting the spark-doris-connector JAR contents and identifying which Spark JARs and Calcite JARs are bundled. Review the connector's dependency and packaging configuration to determine which dependencies are already provided by Spark and whether Calcite is required. Done means producing a smaller connector JAR without unnecessary duplicate dependencies and verifying that it still works with its supported Spark environment.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spark
- Domain
- build-system, data-engineering
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100