Finish the cutover to source-generated schemas and publish missing version folders
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 523
- Forks
- 75
- Avg merge
- 3d 16h
- Merged PRs (30d)
- 24
Description
Summary of the new feature / enhancement
As a configuration author or resource developer using the published DSC schemas,
I want the schemas to publish for every released version and stay in sync with the engine automatically,
So that editor validation and IntelliSense help me instead of contradicting DSC.
Currently the published schemas under schemas/ are built from hand-maintained YAML sources (schemas/src via schemas/build.ps1) that were last generated for v3.1.0, while the engine has moved on to 3.4.
The most visible symptom is that recognized $schema URIs don't resolve. The engine accepts every version through v3.2.3 as a valid $schema value, but no schema folder was ever published after v3.1.0, so a document pinned to a URI like this validates in DSC and 404s in the editor:
$schema: https://aka.ms/dsc/schemas/v3.2/bundled/config/document.json
The YAML sources have also drifted from the engine — fields, enum values, and whole subsystems added since v3.1.0 aren't represented, and in several cases the published schemas reject manifests this repository ships. Those content gaps will be filed as individual issues; this issue tracks the underlying problem that makes them inevitable.
That underlying problem is that the repository has two schema systems mid-migration. The YAML pipeline is what publishes, but every release requires manually editing schemas.config.yaml, re-running build.ps1 per version folder, and hand-updating the $schema URI enums in the sources — steps nothing enforces, which is how seven releases shipped without schemas. Meanwhile the source-based generator from #538/#1406 (DscRepoSchema + cargo xtask schema export) derives schemas directly from the engine's types — making drift structurally impossible — but is nearly complete rather than complete: it has never published anything, and no CI exercises either system.
Proposed technical implementation details (optional)
Finish the cutover to generated schemas in phases, each independently reviewable:
- Exporter correctness (#1672): fix the two silent path collisions (
AdaptedDscResourceManifest,DeleteWhatIfResult), export every type that derivesDscRepoSchema, fail on duplicate output paths, add--schema-version/--releasetargeting, and routedsc schemathrough the same machinery so the CLI and exporter agree. - Bundling and namespace parity: expand
should_bundleto match the 24 bundled schemas the YAML pipeline publishes today, and decide the disposition of each YAML-only schema (resource/stdout/*,resource/properties/*,metadata/Microsoft.DSC/*), with$refalias stubs in the floating version folders for moved paths. - Authoring parity: port the
title/description/VS Code keywords from the YAML sources onto the Rust types using the existingschema_i18n!pattern, with a comparison report gating the cutover so generated schemas don't regress the editor experience. - CI: commit
schemas/vNext, regenerate it in CI and fail on drift, and validate the repository's own example configurations and manifests against the generated schemas. - Backfill: publish the missing
v3.1.1–v3.2.3folders from each release tag's own sources so every recognized URI resolves. - Cutover: the next stable release publishes its folders via
cargo xtask schema export --release, after whichschemas/srcandschemas/build.ps1can be retired.
The CI checks (phase 4) fit naturally as a Pester test in dsc/tests, so they run in the existing ./build.ps1 -Test -PesterTestGroup dsc job as well as in a dedicated schemas workflow that invokes cargo xtask schema export first. Something like:
BeforeDiscovery {
# Every (folder_path, base_name) pair declared with #[dsc_repo_schema(...)]
$declarations = Get-ChildItem lib/dsc-lib/src -Recurse -Filter '*.rs' | ForEach-Object {
[regex]::Matches(
(Get-Content $_.FullName -Raw),
'(?s)base_name\s*=\s*"(?<base>[^"]+)"\s*,\s*folder_path\s*=\s*"(?<folder>[^"]+)"'
) | ForEach-Object {
@{ folder = $_.Groups['folder'].Value; base = $_.Groups['base'].Value }
}
}
}
Describe 'exported schemas' {
It 'exports <folder>/<base>.json' -TestCases $declarations {
Join-Path 'schemas/vNext' $folder "$base.json" | Should -Exist
}
It 'has no uncommitted drift' {
git diff --exit-code -- schemas/vNext
$LASTEXITCODE | Should -Be 0 -Because 'run `cargo xtask schema export` and commit the result'
}
}
Describe 'instance validation' {
It 'validates <name> against the generated manifest schema' -TestCases $manifests {
Test-Json -Path $path -SchemaFile 'schemas/vNext/bundled/resource/manifest.json' | Should -BeTrue
}
}
The first block also catches the "derives DscRepoSchema but was never exported" class of bug statically, and the instance validation is the regression proof for the drift issues: manifests in this repository that fail against the published v3.1.0 schemas today must pass against the generated ones.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading schemas/src and schemas/build.ps1, then inspect the DscRepoSchema implementations under lib/dsc-lib and run cargo xtask schema export. Compare the generated output with the existing schemas/ folders and the proposed Pester coverage in dsc/tests. Done means generated schemas cover the declared types and released versions, avoid drift, and validate the repository's example manifests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- powershell, rust
- Domain
- build-system, ci-cd, devtools
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100