Run REX documentation validation in public PR pipelines
- Dominant language
- C#
- Stars
- 135
- Forks
- 260
- Avg merge
- 3d 1h
- Merged PRs (30d)
- 143
Description
## Goal
Run REX documentation generation against packages built by public pull-request pipelines so malformed API doc comments are caught before nightly docs builds or release.
REX is already used during docs publishing:
- Java: `java2docfx`
- Python: `py2docfx`
- JavaScript: `@microsoft/type2docfx`
The tools are public, but current validation is coupled to `Update-DocsMsMetadata.ps1` and normally downloads an already-published package. PR validation must instead use the package artifacts produced by the current build.
## Work
1. **Extract reusable validation orchestration in `eng/common`**
- Add a script/pipeline template that reads `PackageInfo` files, invokes the language-specific `Validate--DocMsPackages` implementation, aggregates failures, and fails with the affected package names.
- Keep docs-repository checkout and metadata updates out of this path.
- Reuse the same underlying validation from release/docs publishing to prevent behavior drift.
2. **Support local package artifacts in each language repository**
- **Java:** run `java2docfx` with package coordinates and the staged local Maven package folder.
- **Python:** configure `py2docfx` with a local wheel/source distribution (`dist_file`) and make other PR-built wheels available for dependencies.
- **JavaScript:** pass the staged `.tgz` directly to `type2docfx`, ensuring packages changed together can resolve one another.
3. **Add public PR validation jobs**
- Java: depend on `Build`, download `packages`.
- Python: depend on `Build_Extended`, download `packages_extended`.
- JavaScript: depend on `Build`, download `packages`.
- Discover affected packages from generated `PackageInfo`, since `Service=auto` determines the package set dynamically.
- Match existing docs eligibility rules, including `skipPublishDocMs`, private/non-publishable packages, and Java implementation-package exclusions.
- Missing expected artifacts should fail as infrastructure errors; no eligible packages should be a successful no-op.
4. **Test and roll out**
- Add tests for local-artifact selection, multiple-package aggregation, skipped packages, no-op behavior, and missing artifacts.
- Use public test PRs per language to prove a valid package passes, a malformed doc comment fails, and fixing only the comment passes.
- Start non-blocking to identify false positives and measure duration, then make the check required.
- Provide a pipeline-variable kill switch for rapid rollback.
## Done when
Public Java, Python, and JavaScript pull-request builds run REX against the exact packages built by the PR, report package-specific failures, require no internal feed credentials, and block malformed documentation before merge.
Contributor guide
Assessment
This issue has not been assessed yet.