Azure / Azure/typespec-azure

Best practice for smoke testing generated code after emitter move to typespec-azure

Open
#4,599 4 comments 0 reactions 1 assignee Claimed by @msyyc View on GitHub
emitter:client:all
Dominant language
TypeScript
Stars
27
Forks
90
Avg merge
1d 22h
Merged PRs (30d)
156

Description

## Context

When `@azure-tools/typespec-ts` lived in [Azure/autorest.typescript](https://github.com/Azure/autorest.typescript), we had a sibling package `typespec-test` that served as a **smoke test**: it ran the emitter against a large suite of TypeSpec specs and committed the full generated TypeScript output into the repo. This gave us several practical benefits during day-to-day development:

- **Diffable generated code** — any emitter change produced a visible diff in PRs, making regressions and unintended changes obvious at review time.
- **Easy local debugging** — contributors could open the generated client code directly, set breakpoints, inspect output, and iterate on the emitter quickly.
- **End-to-end coverage** — caught build/compile/format issues in generated code that unit tests and Spector tests don't surface.

After the emitter moved to [Azure/typespec-azure](https://github.com/Azure/typespec-azure/tree/main/packages/typespec-ts), `typespec-test` was **not** moved along with it because of the size of the committed generated code. As a result, we've lost the convenient "see the diff, debug the output" workflow in the new repo.

## Discussion

I'd like to hear from other language emitter owners (Python, Java, Go, etc.) on how they handle this:

- Do you keep a smoke-test-style fixture repo with committed generated code? Where does it live?
- If not, how do you review the impact of an emitter change on real-world output before merging?
- How do contributors locally debug generated code against an in-development emitter?
- Are there shared tools or conventions in `typespec-azure` we could adopt instead of each language reinventing one?

## Goal

Agree on a best practice for TypeScript (and potentially other languages) so that:

1. PR reviewers can still see the impact of emitter changes on generated code.
2. Contributors retain a fast inner-loop for debugging generated output.
3. We don't bloat the main repo with megabytes of generated code.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.