apache / apache/pinot

Refactor Segment Name Generator

Open
#8,799 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
2d 3h
Merged PRs (30d)
195

Description

Currently there are three issues with `SegmentNameGenerator` classes -

* The implementations need to be supplied with exact arguments instead of general segment name generator config. e.g. You need to provide `dateTimeFormatSpec`, `pushType` , `excludeSequenceId` etc. but instead you can simply pass `segmentNameGeneratorConfig` to the constructor and let the implementation take care of extracting out props.

* Any change to segmentName logic requires changing the implementation classes. We can rely on something like templates to generate the segment name rather than having to implement code change.

* A lot of segment name generator props are duplicates in `BatchConfigProperties` as well. Addressing 1 should also help addresses this.

Contributor guide

Open the contributing guide

Research direction

Start by locating the SegmentNameGenerator classes and BatchConfigProperties, then compare their constructor arguments and duplicated properties. Review how segmentNameGeneratorConfig is currently consumed and how segment-name logic is implemented. Done means implementations accept the general config, segment naming can be driven by templates, and duplicated properties are addressed.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.