apache / apache/pinot

Time column transform with same name

Open
#6,517 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
1d 21h
Merged PRs (30d)
189

Description

It will be useful to have a transform function in the ingestion path that transforms the time column into different units (e.g. mills to hours since epoch, or millis to minutes since epoch, etc.) without needing to change the column name.

In fact, this functionality is useful for any column.

That way, if an application has data in hadoop, or is ingesting data into a stream, it can use the same column names in Pinot as well.

Contributor guide

Open the contributing guide

Research direction

Start by locating Pinot's ingestion-path transformation entry points and any existing time or column transformation tests. Determine how a transformed value can retain its original column name and how units are specified. Done means ingestion supports time-unit conversion and the same-name transformation behavior for other columns, with coverage for the requested cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.