Enable SDK Smoke Tests to Validate Generated Code Changes
- Dominant language
- TypeScript
- Stars
- 27
- Forks
- 90
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 156
Description
### Background
Today, most validation focuses on payload correctness through Spector tests. While Spector is valuable for validating API behaviors and generated payloads, it does not provide sufficient coverage for validating the generated SDK code when upstream changes occur in TypeSpec, TCGC, emitters, or related code generation components.
As code generation evolves, seemingly small upstream changes can introduce unintended SDK code differences that are difficult to detect until later in the development cycle.
As we all emitters into one repo, here are a couple of challenges:
- Emitter developers have limited visibility into how changes affect generated SDK outputs.
- Impact analysis across TypeSpec, TCGC, emitters, and shared tooling is slow and inconsistent.
- AI-assisted development and automated code changes lack fast feedback signals to validate generated SDK quality.
### Actions:
- Create a lightweight SDK smoke test framework focused on generated code validation.
- Select a small set of representative services (~5 services) covering major generation scenarios.
- Automatically regenerate SDKs when TypeSpec, TCGC, emitter, or shared code generation changes occur.
- Extend Contoso service to cover common generation scenarios and use it as a primary validation service.
- Detect and surface meaningful SDK code differences and integrate smoke tests into CI pipelines and developer workflows.
Contributor guide
Assessment
This issue has not been assessed yet.