apache / apache/datafusion-comet

feat: Add a version method to the configuration information of CometConf

Open
#3,700 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Scala
Stars
1.3k
Forks
373
Avg merge
2d 4h
Merged PRs (30d)
198

Description

Add a version method to the configuration information of CometConf.

Eg:
COMET_IO_ADJUST_READRANGE_SKEW

after this pr:

val COMET_IO_ADJUST_READRANGE_SKEW: ConfigEntry[Boolean] =
conf("spark.comet.parquet.read.io.adjust.readRange.skew")
.category(CATEGORY_PARQUET)
.version("0.14.0")
.doc("In the parallel reader, if the read ranges submitted are skewed in sizes, this " +
"option will cause the reader to break up larger read ranges into smaller ranges to " +
"reduce the skew. This will result in a slightly larger number of connections opened to " +
"the file system but may give improved performance.")
.booleanConf
.createWithDefault(false)

Contributor guide

Open the contributing guide

Research direction

Locate CometConf and the ConfigEntry builder definitions, then inspect how existing configuration metadata methods are implemented. Verify that the shown COMET_IO_ADJUST_READRANGE_SKEW declaration can call version("0.14.0") and that the version is retained in its configuration information.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala
Domain
tooling
Issue type
Feature
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.