openssl / openssl/openssl

RSA based certs failing during TLS after removal of RSA-PSS ciphers from ClientSignatureAlgorithm

Open
#28,942 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

issue: bug report
Dominant language
C
Stars
30.8k
Forks
11.5k
Avg merge
10m
Merged PRs (30d)
1

Description

For FIPS mode, I added ClientSignatureAlgorithm in opensslcnf.txt. This change was done for FIPS mode using the crypto-policies package.
The support ciphers for ClientSignatureAlgorithms are same as SignatureAlgorithms with these ciphers removed:

ecdsa_secp521r1 with sha512(0x0603),
rsa_pss_rsae with sha256(0x0804),
rsa_pss_rsae with sha384(0x0805),
rsa_pss_rsae with sha512(0x0806),

We're facing issues while connecting to some update servers after this change was done and this is the error log:

OpenSSL SSL_read: error:1409445C:SSL routines:ssl3_read_bytes:tlsv13 alert certificate required, errno 0

With RSA-PSS ciphers removed, it still contains multiple RSA-based(RSA+SHA256:RSA+SHA384:RSA+SHA512) ciphers for negotiation and ideally this should work fine but that's not the case:

Opensslcnf.txt:

CipherString = @SECLEVEL=2:kEECDH:kEDH:kPSK:kDHEPSK:kECDHEPSK:!kRSA:-aDSS:!CHACHA20-POLY1305:-3DES:!DES:!RC4:!RC2:!IDEA:-SEED:!eNULL:!aNULL:-AESCCM:!MD5:-SHA384:-CAMELLIA:-ARIA:-AESCCM8
Ciphersuites = TLS_AES_256_GCM_SHA384:TLS_AES_128_GCM_SHA256
MinProtocol = TLSv1.2
MaxProtocol = TLSv1.3
SignatureAlgorithms = ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:rsa_pss_rsae_sha256:rsa_pss_rsae_sha384:rsa_pss_rsae_sha512:RSA+SHA256:RSA+SHA384:RSA+SHA512
ClientSignatureAlgorithms = ECDSA+SHA256:ECDSA+SHA384:RSA+SHA256:RSA+SHA384:RSA+SHA512            <== Newly added

OpenSSL Version - OpenSSL 1.1.1g FIPS

  • OpenSSL build with a few severity patches (official Red Hat ones)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the ClientSignatureAlgorithms and SignatureAlgorithms settings in opensslcnf.txt and the TLS 1.3 certificate-required error in the report. Review how OpenSSL 1.1.1g FIPS handles RSA certificate negotiation when RSA-PSS algorithms are excluded, using the supplied configuration and update-server connection as the reproduction. Done means the cause is identified and RSA-based certificate connections work without weakening the stated FIPS policy.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
cryptography, networking, security
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.