microsoft / microsoft/aspire

Playwright CLI provenance verification fails (`AttestationFetchFailed`) and blocks `aspire new` when `registry.npmjs.org` is unreachable

Open
#19,159 3 comments 0 reactions 0 assignees View on GitHub
area-cli
Dominant language
C#
Stars
6.3k
Forks
991
Avg merge
2d 15h
Merged PRs (30d)
196

Description

## Summary

During 13.5 dogfooding, running `aspire new` (which triggers agent-environment configuration / `aspire agent init`) fails to install the `@playwright/cli` skill with:

```
❌ Provenance verification failed for @playwright/cli@0.1.17: AttestationFetchFailed
⚠️ Configuration completed with errors. Please fix the reported issues and re-run the command.
```

The command completes with errors even though everything else (template creation, skill install, telemetry hooks) succeeds.

**CLI version:** `13.5.0-preview.1.26405.3+0098d40c398009ca333f274a1de58ab2ca61a069`

> Note: the provenance verification code has existed since v13.4.6, so this is not strictly a brand-new 13.5 regression — but it is an active, high-impact bug hit during 13.5 dogfooding, so tracking it under 13.5.

## Root cause

The Playwright CLI installer resolves, downloads (`npm pack`), and SHA-512 hash-pins the tarball from the **trusted internal Azure Artifacts feed** (`dotnet-public-npm` on `pkgs.dev.azure.com/dnceng`). That part works fine.

But provenance verification (`SigstoreNpmProvenanceChecker`) then fetches the attestation bundle from a **hardcoded public endpoint**:

```
https://registry.npmjs.org/-/npm/v1/attestations/@playwright/cli@{version}
```

In locked-down / corporate / air-gapped environments — exactly the environments that point npm at the internal ADO mirror because public npm is blocked — this request fails. In the repro below it fails with a TLS handshake failure:

```
System.Net.Http.HttpRequestException: The SSL connection could not be established
---> System.Security.Authentication.AuthenticationException: Authentication failed because the remote party sent a TLS alert: 'HandshakeFailure'.
```

`FetchAttestationJsonAsync` returns `null` → `VerifyProvenanceAsync` returns `ProvenanceVerificationOutcome.AttestationFetchFailed` → `PlaywrightCliInstaller` treats it as a **hard failure** (`PlaywrightInstallStatus.Failed`), which surfaces as "Configuration completed with errors."

This is a **known design gap**, already documented in `docs/specs/safe-npm-tool-install.md` under *Future Improvements*:

> **Internal attestation mirror** — Preserve npm's DSSE attestation response in an internal service so provenance verification does not require a public registry request. Azure Artifacts currently mirrors the tarball and SHA-1 metadata but omits the SHA-512 integrity value and attestation bundle.

Confirmed: the internal ADO feed returns **HTTP 404** for the attestations API path, so there is no in-network fallback today.

## Reproduction

On a machine that cannot reach `registry.npmjs.org` (e.g. npm registry configured to the internal `dotnet-public-npm` feed, public npm TLS-blocked):

```powershell
aspire new
# Select any template, answer "Y" to "configure AI agent environments",
# and include the playwright-cli skill.
```

Minimal repro of just the failing network call:

```powershell
curl.exe -sS "https://registry.npmjs.org/-/npm/v1/attestations/%40playwright%2Fcli@0.1.17"
# curl: (35) schannel: ... SEC_E_ILLEGAL_MESSAGE (0x80090326) - fatal SSL/TLS alert (handshake failed)
```

(The internal feed the tarball came from returns 404 for the same attestations path, so it can't serve as a fallback.)

## Impact

- `aspire new` / `aspire agent init` reports failure and exits with errors for **any user whose network cannot reach `registry.npmjs.org`** — including many enterprise/corporate networks and the Microsoft-internal dogfooding environments that use the internal ADO feed.
- A **transient/environmental** connectivity failure is treated identically to a genuine provenance mismatch, even though the package was already safely resolved and hash-pinned from the trusted internal feed.

## Suggested directions (for discussion)

1. **Fail soft on fetch/transport errors:** distinguish `AttestationFetchFailed` (network/transport) from real verification failures (`SourceRepositoryMismatch`, `SlsaProvenanceNotFound`, signature failures). When attestations simply can't be fetched, emit a clear warning and continue rather than blocking configuration — the tarball is already integrity-pinned from the trusted internal feed.
2. **Internal attestation mirror** (the spec's own "Future Improvement") so verification doesn't depend on a public request.
3. At minimum, make the failure **non-fatal for `aspire new`** and/or surface the existing `disablePlaywrightCliPackageValidation` break-glass key more prominently in the error message.

## Relevant code

- `src/Aspire.Cli/Npm/SigstoreNpmProvenanceChecker.cs` — `FetchAttestationJsonAsync`, `NpmRegistryAttestationsBaseUrl = "https://registry.npmjs.org/-/npm/v1/attestations"`
- `src/Aspire.Cli/Agents/Playwright/PlaywrightCliInstaller.cs` — treats `!provenanceResult.IsVerified` as `PlaywrightInstallStatus.Failed`
- `docs/specs/safe-npm-tool-install.md` — threat model + "Internal attestation mirror" future improvement

Contributor guide

Open the contributing guide

Research direction

Start with src/Aspire.Cli/Npm/SigstoreNpmProvenanceChecker.cs, especially FetchAttestationJsonAsync and NpmRegistryAttestationsBaseUrl, then trace the result handling in src/Aspire.Cli/Agents/Playwright/PlaywrightCliInstaller.cs. Reproduce with the documented aspire new flow or attestation curl request; done means unreachable registry.npmjs.org no longer causes an inappropriate configuration failure while genuine provenance failures remain distinguishable.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, nodejs
Domain
cli, networking, security
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.