apache / apache/pinot

Allow transform function as the dimension in the star-tree

Open
#8,863 1 comment 0 reactions 0 assignees View on GitHub
feature
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
1d 21h
Merged PRs (30d)
189

Description

Currently in order to build a star-tree on top of a transform function, user needs to follow these steps:
1. Create a derived column (single-value dictionary encoded) from the transform
2. Create star-tree on top of the derived column
3. Query derived column in order to use star-tree

We should be able to simplify these steps and allow transform function as the dimension in star-tree. Internally we can follow the same steps and generate a hidden column for the derived column and build star-tree on top of it. At query time, we may re-write the query to query the hidden column

Contributor guide

Open the contributing guide

Research direction

Start by tracing the star-tree construction path and the query-rewrite path for transform functions. Verify how a derived column is currently created and used, then determine the changes needed for an internal hidden column. Done means users can declare a transform function directly as a star-tree dimension and queries use the resulting star-tree without referencing the derived column.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.