apache / apache/pinot

schema compatibility check implementation is too restrictive

Open
#8,945 3 comments 1 reaction 0 assignees View on GitHub
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
1d 21h
Merged PRs (30d)
189

Description

Version: 0.10.0

Current implementation is too restrictive. It almost allows no change other than column addition.
Few examples which we came across after upgrade (0.10.0 from 0.7.1)
- Change in default value
- Change from dimension to metric
- Change in granularity of a DateTimeField
- Changing the maxLength of a column

These changes were allowed (due to a bug however) in 0.7.1
Checking for compatibility is very important. But, making it too restrictive also makes it unusable and something we need to fix I believe.

More discussion and details in this thread.
https://apache-pinot.slack.com/archives/CDRCA57FC/p1655230812461749

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the schema compatibility implementation and the linked Slack discussion for the expected behavior. Done means compatibility checks accommodate the listed default, dimension/metric, DateTimeField granularity, and maxLength changes without making validation unusable.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.