microsoft / microsoft/vscode-azureresourcegroups
Scaffold agent omits `allowedHosts: true` from vite.config, stalling the frontend preview approval gate
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 78
- Forks
- 55
- Avg merge
- 16h 49m
- Merged PRs (30d)
- 139
Description
Summary
The scaffold agent produced a working full-stack project and then shipped a vite.config.ts without allowedHosts: true — the one setting the product's own instructions say is required for the frontend preview webview to load.
This was caught by the MSBench eval suite on its first run against real agent output (scaffold-fullstack, run 2026082620153444, 6/7, resolved: false). The defect was not present in any certification fixture.
Why this is a product bug and not an over-strict gate
I verified the gate's claim against the product's own rails documents before filing. The requirement is stated four times, with the consequence spelled out:
resources/agents/azure-project-scaffold/references/frontend-preview-steps.md:32
In
vite.configsetserver: { host: true, allowedHosts: true, strictPort: false }…allowedHosts: truestops the dev server 403-blocking the forwarded/webview origin
resources/agents/azure-project-scaffold.agent.md:53
the frontend you scaffolded MUST be embeddable in that iframe: its
vite.configserversetshost: true+allowedHosts: true+strictPort: false… If the preview can't load, the user cannot click Approve UI and the whole flow stalls.
Also at resources/agents/azure-project-scaffold/instructions.md:173 and resources/agents/shared-references/architecture.md:391.
So this is not a case of the eval inventing a requirement. The agent violated an explicitly documented, repeatedly stated contract, and the documented consequence is a hard stall of the guided flow: the preview webview fails to load, the user cannot click Approve UI, and the scaffold→integrate hand-off never happens.
Evidence
FAIL: scaffolded frontend is preview-embeddable and keeps the API seam
• [devServerRejectsWebviewOrigin] services/web/vite.config.ts:
vite.config server must set `allowedHosts: true` or the dev server
403-blocks the webview origin.
Run 2026082620153444 · scaffold-fullstack · completed 14m34s · exit 1.
Attribution checked against all four known traps before believing the red:
| Trap | Status |
|---|---|
NOT_APPLICABLE / coverageGap marker on stderr |
absent |
| exit 3 (grader itself broke) | no — exit 1 |
RATE_LIMIT / -> 429 |
no error.json at all |
timeoutMs kill |
no — whole run 14m34s |
Everything else in the run passed, which is what makes the red meaningful rather than noise:
validate-project-builds --require-frontend— passed. The project genuinely installs and builds.validate-integration-plan --has-frontend— passed.- Frontend preview approval gate opened; no premature hand-off to integrate; no chat-question fallback; liveness sentinel passed.
So the agent did almost everything right, including the expensive part, and missed one line of config that stalls the user-visible flow.
Suggested direction
Worth considering why an explicitly and repeatedly documented requirement was missed, rather than only fixing the instance:
- The requirement lives in a reference document (
frontend-preview-steps.md) that the agent may not read on every path, whileinstructions.md:173mentions it inside a long checklist item. - A generated
vite.config.tsis a small, high-leverage artifact — a template or a post-scaffold check would enforce this more reliably than prose. The product already knows the exact required shape:server: { host: true, allowedHosts: true, strictPort: false }. - The same section lists sibling requirements with identical stall consequences (
devscript must actually serve; noX-Frame-Options/frame-ancestorsCSP; must not start a competing dev server). If prose is not sufficient forallowedHosts, it is likely not sufficient for those either — and the eval gate covers them, so we will find out.
Provenance
Found by the scaffold-fullstack eval stimulus on feat/CoR. This is the first defect the suite has found in real product output rather than in a fixture — the gate that caught it had been mutation-tested but never before run against a live agent.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with resources/agents/azure-project-scaffold.agent.md, instructions.md, and references/frontend-preview-steps.md, then inspect the generated services/web/vite.config.ts. Run the scaffold-fullstack evaluation or its preview-embeddability gate, using run 2026082620153444 as the failing case. Done means the generated frontend passes the allowedHosts check and the preview webview reaches the Approve UI gate.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript, vite
- Domain
- frontend, tooling
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 70/100