Route PKI CRL and denylist fetching through the hardened HTTP client

Context

PR #4420 hardens the shared strict-mode HTTP client against SSRF:

• A dial guard on client.SafeHttpTransport blocks connections to non-public addresses in strict mode. It runs after DNS resolution, so it also catches hostnames that resolve into internal ranges. Operators can exempt ranges with http.client.allowedinternalcidrs.
• StrictHTTPClient refuses non-HTTPS first hops, refuses redirects that downgrade from HTTPS to HTTP, and caps response bodies at 1MB.

These protections are split across two layers on purpose. The transport carries the invariant: never dial a non-public address in strict mode. The client wrapper carries policy that is correct for federation traffic: HTTPS only, no downgrade on redirect, bounded small bodies.

Gap

The PKI package builds its own HTTP clients and bypasses both layers:

• pki/validator.go (newValidator): CRL fetching uses http.DefaultTransport directly.
• pki/denylist.go (download): denylist sync uses a bare http.Client.

Both carry a comment saying the safe client is not needed because the resource is trusted. For the denylist that mostly holds: the URL is operator-configured (default is the nuts-foundation denylist on a public CDN) and the payload signature is verified. For CRLs it does not hold: CRL distribution point URLs come out of certificates. A misissued or compromised participant certificate can point its CDP anywhere, including an internal address, and the node will fetch it. In strict mode that is exactly the SSRF vector the dial guard from #4420 closes for federation traffic.

Why StrictHTTPClient does not fit as-is

Two of the client-tier policies are wrong for CRL fetching:

1. CRL distribution points are commonly plain HTTP by design. The CRL is signed, and requiring HTTPS creates a bootstrap problem (checking revocation of the certificate that protects the revocation download). The HTTPS check would break CRL fetching for real CAs.
2. Real-world CRLs exceed the 1MB response cap.

The transport-tier invariant (the dial guard) fits without reservation.

Proposal

1. Add functional options to the http/client constructors so the client-tier policy becomes configurable per use case:

   • WithInsecureScheme(): permits plain HTTP. Keeps the 10-redirect cap, drops the downgrade check (meaningless when the first hop may be plaintext). The name is deliberately unattractive so misuse stands out in review.
   • WithMaxResponseSize(n): keeps response bodies bounded but configurable (for example 20MB for CRLs).

2. Migrate pki/validator.go and pki/denylist.go to these constructors, for example:

   client.New(syncTimeout, client.WithInsecureScheme(), client.WithMaxResponseSize(20<<20))
   

   This inherits the dial guard, the http.client.allowedinternalcidrs allowlist, the TLS 1.2 minimum, and the shared tracing wrapper. The hand-wired otelhttp setup in newValidator can then be removed.

3. Document on WithInsecureScheme what it is for: content that carries its own integrity protection (signed CRLs, the signed denylist), never federation endpoints.

Operational impact

• Public deployments: no change expected. Public CA CRL endpoints and the default denylist URL resolve to public addresses.
• Closed or strict-mode deployments where a private CA publishes CRLs on an internal address: after this change the dial guard blocks CRL refresh. With the default pki.softfail=true the node keeps accepting connections while revocation data silently goes stale. With pki.softfail=false connections start failing once the data is stale (pki.maxupdatefailhours, default 4). This is a time-delayed failure mode, so it needs an explicit release note: such deployments must add their internal ranges to http.client.allowedinternalcidrs.

Out of scope

• Storage clients (Vault, external key store, SQL, Redis): deliberately excluded in #4420. They legitimately talk to private addresses and do not fetch URLs influenced by remote parties.

Follow-up to #4420.

Start with the HTTP client constructors and the existing policy layers, then read pki/validator.go (newValidator) and pki/denylist.go (download) to trace their current clients and tracing setup. The work is done when both fetch paths use the shared constructors with appropriate options, the documented security boundary is preserved, and the strict-mode operational impact has an explicit release note.