apache / apache/incubator-xtable
Generate a JSON Schema for datasetConfig from DatasetConfig
- Dominant language
- Java
- Stars
- 1.2k
- Forks
- 212
- Avg merge
- 4d 9h
- Merged PRs (30d)
- 16
Description
### Feature Request / Improvement
Publish a JSON Schema for `datasetConfig.yaml`, generated from the `DatasetConfig` class rather than written by hand.
## Why
Writing a valid `datasetConfig` is the first thing a new user does and the first place they get stuck: `sourceFormat`, `targetFormats`, `tableBasePath`, `tableName`, `namespace`, `partitionSpec`, and the right `conversionSourceProviderClass`. Today the only description of that shape is prose in the docs, so it drifts from the code silently.
A generated schema gives three things at once:
- Editors and IDEs validate and complete the file.
- Any agent or script can check a config before running a sync, instead of discovering problems from a stack trace.
- The documentation stops being a second source of truth that can go stale.
PR #829 includes a hand-written `config-schema.md` for exactly this purpose. That is the thing to replace with something generated.
## Scope
- Generate a JSON Schema from `DatasetConfig` and its nested types at build time.
- Publish it as a release artifact and at a stable URL, so `$schema` references keep working.
- Fail the build when the schema is out of date with the model, so it cannot silently drift.
- Reference it from the docs in place of any hand-maintained field list.
### Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the DatasetConfig class and its nested types, then inspect the hand-written config-schema.md from PR #829 and the datasetConfig.yaml documentation. Define how the build generates and validates the schema, publishes it as a release artifact at a stable URL, and replaces the maintained field list in the docs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- build-system, documentation, release
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100