schema compatibility check implementation is too restrictive
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 189
Description
Version: 0.10.0
Current implementation is too restrictive. It almost allows no change other than column addition.
Few examples which we came across after upgrade (0.10.0 from 0.7.1)
- Change in default value
- Change from dimension to metric
- Change in granularity of a DateTimeField
- Changing the maxLength of a column
These changes were allowed (due to a bug however) in 0.7.1
Checking for compatibility is very important. But, making it too restrictive also makes it unusable and something we need to fix I believe.
More discussion and details in this thread.
https://apache-pinot.slack.com/archives/CDRCA57FC/p1655230812461749
Contributor guide
Research direction
Start by reviewing the schema compatibility implementation and the linked Slack discussion for the expected behavior. Done means compatibility checks accommodate the listed default, dimension/metric, DateTimeField granularity, and maxLength changes without making validation unusable.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100