Handling incomplete certificate chains in Node TLS

What is the problem this feature will solve?

Servers should return a complete certificate chain, which can be validated up to a trusted root.

Sadly, some don't, and instead return a chain that references an intermediate cert signed by a trusted root, but doesn't actually include the intermediate. There's also possible cases where an intermediate cert expires, and the authority has reissued a new intermediate with the same key, but the chain only contains the old intermediate.

There's a test site for this here: https://incomplete-chain.badssl.com/. You can open this in your browser just fine, but in Node:

> require('https').request('https://incomplete-chain.badssl.com/')
...
Uncaught Error: unable to verify the first certificate
    at TLSSocket.onConnectSecure (node:_tls_wrap:1679:34)
    at TLSSocket.emit (node:events:518:28)
    at TLSSocket.emit (node:domain:552:15)
    at TLSSocket._finishInit (node:_tls_wrap:1078:8)
    at ssl.onhandshakedone (node:_tls_wrap:864:12)
    at TLSWrap.callbackTrampoline (node:internal/async_hooks:130:17) {
  code: 'UNABLE_TO_VERIFY_LEAF_SIGNATURE',

Incomplete chains like this are bad behaviour, but it's also more common than you'd think, because it works in most places. More specifically: all modern browsers (Chrome, Edge, Safari, FF) and Mac/Windows OS libraries (Secure Transport & schannel) all seem to handle this automatically.

These missing intermediates are generally handled with one a few different approaches:

• Caching intermediate certs seen elsewhere, so you can validate any subsequent certificates that reference this intermediate, even if they don't include it.
• Reading Authority Information Access (AIA) metadata from the certificates that are provided, to dynamically fetch missing intermediate certs when required.
• Preloading commonly used intermediates directly - effectively starting with a complete cache.

AFAICT Secure Transport (macOS), schannel (Windows), Chrome, Edge & Safari all use AIA fetching to handle this, while Firefox uses intermediate preloading (more details: https://wiki.mozilla.org/Security/CryptoEngineering/Intermediate_Preloading).

Python's similar discussion about this may be interesting: https://github.com/python/cpython/issues/62817. There is an OpenSSL issue about this but zero activity: https://github.com/openssl/openssl/issues/27016.

Currently Node has no good way to solve this problem. It's not handled automatically, but it's also not really very practical to handle manually either unless you disable TLS validation entirely and do it all yourself in userspace (not a good idea).

What is the feature you are proposing to solve the problem?

Assuming people are on board with trying to offer a solution here, there's a few initial questions:

• Is there interest in Node trying to handle this transparently for users, so everything that succeeds in a browsers succeeds in Node? Or should we just offer an API to make it easy to handle in userspace?
• How do we feel about AIA vs intermediate preloading? Firefox has gone for the latter, Python seems to be leaning that way too, but Chrome et al or the OS implementations are all on the AIA train (AFAICT).

From a quick exploration of the options, to me it looks like there's no easy way to support AIA within OpenSSL today, so AIA would imply connecting, failing, doing AIA fetching if it might help, and then connecting again. That could be done transparently in Node, or in userspace if we exposed enough cert info in errors for users to fetch & retry this themselves.

It is possible to include extra intermediate certificates with OpenSSL, by using X509_STORE_CTX_set0_untrusted to add certificates that can be used to build a chain, but which aren't actually trusted in themselves (but we don't currently use or expose this to JS anywhere). That could be used to implement preloading, caching, or the connection-retry AIA fetching approach.

Would love to hear thoughts from @nodejs/crypto.

What alternatives have you considered?

No response

Read the TLS failure path identified by node:_tls_wrap and the issue’s references to X509_STORE_CTX_set0_untrusted; compare the AIA, caching, and preloading options with the crypto discussion. Done means an agreed implementation direction, exposed behavior or API, and tests demonstrating validation of incomplete-chain.badssl.com without disabling TLS verification.