Fixing validation step for schema/table codepath
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 195
Description
Currently table config validation are not just triggered when table config is being add/updated. some part of the validation code is also used outside of table config change code path.
several issues we want to fix.
1. ensure that table config validation only runs when table config are being updated/added because it is the only codepath that supports a user manual SKIP option. (see #7921)
2. other types of validation (such as schema validation) should not be triggered when table config is being validate since table config doesn't contain schema.
Additionally, for backward compatibility, we should not run validation against existing, stored configs because it blocks seamless upgrade from older pinot version to newer one.
Contributor guide
Research direction
Start by tracing the table-config validation codepath and the other validation paths described in the issue, using #7921 for the manual SKIP behavior. Verify when schema validation is invoked and whether stored configs are revalidated during upgrades. Done means table-config validation runs only for additions or updates, unrelated schema validation is excluded, and existing stored configs remain unvalidated for backward compatibility.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100