apache / apache/druid

Don't store all-null dimension columns

Open
#6,549 6 comments 0 reactions 0 assignees View on GitHub
Area - Batch Ingestion Performance stale
Dominant language
Java
Stars
14.1k
Forks
3.8k
Avg merge
2d 31m
Merged PRs (30d)
209

Description

When a Druid segment is created, it could be checked that a dictionary-encoded dimension column contains all null values, and just not written to the segment if so.

Probably the same couldn't be done for all-zero numeric columns, because if they are queried as dimensions they should appear as dimensions with all "0" values, not "null" values.

Contributor guide

Open the contributing guide

Research direction

No files, tests, or entry points are named in the issue. Start by tracing Druid segment creation and dictionary-encoded dimension-column writing, then check how omitted all-null columns behave when queried. Done means all-null dimension columns are not written while their query semantics remain correct.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.