Config Simplification
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
EPIC for simplifying configs and improving usability around configs in Hudi
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-5738
- Type: Epic
- Fix version(s):
- 1.1.0
- Attachment(s):
- 29/Feb/24 08:31;geserdugarov;Hudi configuration parameters.xlsx;https://issues.apache.org/jira/secure/attachment/13067149/Hudi+configuration+parameters.xlsx
---
## Comments
29/Feb/24 08:25;geserdugarov;[~guoyihua], [~jonvex], hi!
I'm trying to figure out how to handle huge amount of configuration parameters in Hudi. So, I started from data collecting and want to share current results.
Note, that all numbers are for the master branch, commit d0929341d0ad25b3ce587d7f229d2fcdf7796768 at Feb 26, 2024.
The number of parameters is really huge, current total number is {*}899 parameters{*}.
By modules:
* hudi-flink-datasource - 130 parameters,
* hudi-spark-datasource - 45,
* hudi-client-common - 294,
* hudi-common - 184,
* hudi-utilities - 143,
* kafka, hive, sync, aws, gcp, and datahub related - 103.
740 parameters marked as advanced (only 9 from them are marked as deprecated).
For now, we have 159 not advanced parameters.;;;
---
29/Feb/24 08:31;geserdugarov; [^Hudi configuration parameters.xlsx]
Here my table with all configurations. You can look at all of them, sort, filter, analyze default values.;;;
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.