Time column transform with same name
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 189
Description
It will be useful to have a transform function in the ingestion path that transforms the time column into different units (e.g. mills to hours since epoch, or millis to minutes since epoch, etc.) without needing to change the column name.
In fact, this functionality is useful for any column.
That way, if an application has data in hadoop, or is ingesting data into a stream, it can use the same column names in Pinot as well.
Contributor guide
Research direction
Start by locating Pinot's ingestion-path transformation entry points and any existing time or column transformation tests. Determine how a transformed value can retain its original column name and how units are specified. Done means ingestion supports time-unit conversion and the same-name transformation behavior for other columns, with coverage for the requested cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100