apache / apache/pinot

Fixing validation step for schema/table codepath

Open
#8,046 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
2d 3h
Merged PRs (30d)
195

Description

Currently table config validation are not just triggered when table config is being add/updated. some part of the validation code is also used outside of table config change code path.

several issues we want to fix.
1. ensure that table config validation only runs when table config are being updated/added because it is the only codepath that supports a user manual SKIP option. (see #7921)
2. other types of validation (such as schema validation) should not be triggered when table config is being validate since table config doesn't contain schema.

Additionally, for backward compatibility, we should not run validation against existing, stored configs because it blocks seamless upgrade from older pinot version to newer one.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the table-config validation codepath and the other validation paths described in the issue, using #7921 for the manual SKIP behavior. Verify when schema validation is invoked and whether stored configs are revalidated during upgrades. Done means table-config validation runs only for additions or updates, unrelated schema validation is excluded, and existing stored configs remain unvalidated for backward compatibility.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.