Playwright CLI provenance verification fails (`AttestationFetchFailed`) and blocks `aspire new` when `registry.npmjs.org` is unreachable
- Dominant language
- C#
- Stars
- 6.3k
- Forks
- 991
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 196
Description
## Summary
During 13.5 dogfooding, running `aspire new` (which triggers agent-environment configuration / `aspire agent init`) fails to install the `@playwright/cli` skill with:
```
❌ Provenance verification failed for @playwright/cli@0.1.17: AttestationFetchFailed
⚠️ Configuration completed with errors. Please fix the reported issues and re-run the command.
```
The command completes with errors even though everything else (template creation, skill install, telemetry hooks) succeeds.
**CLI version:** `13.5.0-preview.1.26405.3+0098d40c398009ca333f274a1de58ab2ca61a069`
> Note: the provenance verification code has existed since v13.4.6, so this is not strictly a brand-new 13.5 regression — but it is an active, high-impact bug hit during 13.5 dogfooding, so tracking it under 13.5.
## Root cause
The Playwright CLI installer resolves, downloads (`npm pack`), and SHA-512 hash-pins the tarball from the **trusted internal Azure Artifacts feed** (`dotnet-public-npm` on `pkgs.dev.azure.com/dnceng`). That part works fine.
But provenance verification (`SigstoreNpmProvenanceChecker`) then fetches the attestation bundle from a **hardcoded public endpoint**:
```
https://registry.npmjs.org/-/npm/v1/attestations/@playwright/cli@{version}
```
In locked-down / corporate / air-gapped environments — exactly the environments that point npm at the internal ADO mirror because public npm is blocked — this request fails. In the repro below it fails with a TLS handshake failure:
```
System.Net.Http.HttpRequestException: The SSL connection could not be established
---> System.Security.Authentication.AuthenticationException: Authentication failed because the remote party sent a TLS alert: 'HandshakeFailure'.
```
`FetchAttestationJsonAsync` returns `null` → `VerifyProvenanceAsync` returns `ProvenanceVerificationOutcome.AttestationFetchFailed` → `PlaywrightCliInstaller` treats it as a **hard failure** (`PlaywrightInstallStatus.Failed`), which surfaces as "Configuration completed with errors."
This is a **known design gap**, already documented in `docs/specs/safe-npm-tool-install.md` under *Future Improvements*:
> **Internal attestation mirror** — Preserve npm's DSSE attestation response in an internal service so provenance verification does not require a public registry request. Azure Artifacts currently mirrors the tarball and SHA-1 metadata but omits the SHA-512 integrity value and attestation bundle.
Confirmed: the internal ADO feed returns **HTTP 404** for the attestations API path, so there is no in-network fallback today.
## Reproduction
On a machine that cannot reach `registry.npmjs.org` (e.g. npm registry configured to the internal `dotnet-public-npm` feed, public npm TLS-blocked):
```powershell
aspire new
# Select any template, answer "Y" to "configure AI agent environments",
# and include the playwright-cli skill.
```
Minimal repro of just the failing network call:
```powershell
curl.exe -sS "https://registry.npmjs.org/-/npm/v1/attestations/%40playwright%2Fcli@0.1.17"
# curl: (35) schannel: ... SEC_E_ILLEGAL_MESSAGE (0x80090326) - fatal SSL/TLS alert (handshake failed)
```
(The internal feed the tarball came from returns 404 for the same attestations path, so it can't serve as a fallback.)
## Impact
- `aspire new` / `aspire agent init` reports failure and exits with errors for **any user whose network cannot reach `registry.npmjs.org`** — including many enterprise/corporate networks and the Microsoft-internal dogfooding environments that use the internal ADO feed.
- A **transient/environmental** connectivity failure is treated identically to a genuine provenance mismatch, even though the package was already safely resolved and hash-pinned from the trusted internal feed.
## Suggested directions (for discussion)
1. **Fail soft on fetch/transport errors:** distinguish `AttestationFetchFailed` (network/transport) from real verification failures (`SourceRepositoryMismatch`, `SlsaProvenanceNotFound`, signature failures). When attestations simply can't be fetched, emit a clear warning and continue rather than blocking configuration — the tarball is already integrity-pinned from the trusted internal feed.
2. **Internal attestation mirror** (the spec's own "Future Improvement") so verification doesn't depend on a public request.
3. At minimum, make the failure **non-fatal for `aspire new`** and/or surface the existing `disablePlaywrightCliPackageValidation` break-glass key more prominently in the error message.
## Relevant code
- `src/Aspire.Cli/Npm/SigstoreNpmProvenanceChecker.cs` — `FetchAttestationJsonAsync`, `NpmRegistryAttestationsBaseUrl = "https://registry.npmjs.org/-/npm/v1/attestations"`
- `src/Aspire.Cli/Agents/Playwright/PlaywrightCliInstaller.cs` — treats `!provenanceResult.IsVerified` as `PlaywrightInstallStatus.Failed`
- `docs/specs/safe-npm-tool-install.md` — threat model + "Internal attestation mirror" future improvement
Contributor guide
Research direction
Start with src/Aspire.Cli/Npm/SigstoreNpmProvenanceChecker.cs, especially FetchAttestationJsonAsync and NpmRegistryAttestationsBaseUrl, then trace the result handling in src/Aspire.Cli/Agents/Playwright/PlaywrightCliInstaller.cs. Reproduce with the documented aspire new flow or attestation curl request; done means unreachable registry.npmjs.org no longer causes an inappropriate configuration failure while genuine provenance failures remain distinguishable.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp, nodejs
- Domain
- cli, networking, security
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100