Refactor duplicate function-evaluation paths in ExpressionTransformer and CustomFunctionEnricher
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 195
Description
### Motivation
`pinot-segment-local/src/main/java/org/apache/pinot/segment/local/recordtransformer/ExpressionTransformer.java` and `pinot-segment-local/src/main/java/org/apache/pinot/segment/local/recordtransformer/enricher/function/CustomFunctionEnricher.java` still carry reciprocal TODOs to merge the two implementations.
Both code paths evaluate function-based transformations during ingestion, but they live behind different extension points (`RecordTransformer` vs `RecordEnricher`). That makes behavior drift and duplicate maintenance more likely.
### Scope
- Document the overlap and intended ownership between the two implementations.
- Consolidate shared evaluation logic behind one path or a shared abstraction.
- Preserve current ingestion behavior for transform configs and enricher configs.
- Add or update focused tests for parity and backward compatibility.
### Notes
Observed on `upstream/master` on April 15, 2026.
Contributor guide
Research direction
Start with ExpressionTransformer.java and CustomFunctionEnricher.java, reading the reciprocal TODOs and tracing their RecordTransformer and RecordEnricher extension points. Compare how both paths evaluate function-based transformations, then add or update focused parity and backward-compatibility tests for transform and enricher configurations. Done means shared ownership is clear, duplicate evaluation logic is consolidated, and current ingestion behavior is preserved.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, data-engineering
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 50/100