Make existing intermediate/multi-root CA handling perfect

Status quo

Trusting only external intermediate CA #7719

i.e. making it Icinga's effective root CA.

See https://github.com/Icinga/icinga2/issues/7719#issuecomment-1496177500 . In short, one has to cross-self-sign the intermediate CA and to use the result as Icinga's root CA. Also works w/o sharing CA keys with Icinga if all leaf certs are provided manually.

Using intermediate CAs like everyone else

Icinga-owned intermediate CA

i.e. Icinga signs leaf certs by itself with an intermediate CA in /var/lib/icinga2/ca/ca.* while its root CA is still in /var/lib/icinga2/certs/ca.crt.

Doesn't work, yet. External intermediate CAs work (see below), but at least when it comes to CSR handling, Icinga doesn’t take the whole chain into account.

External intermediate CAs

I'm such sort of a fool. 🙈 https://github.com/Icinga/icinga2/pull/8859#issuecomment-1551364930

I thought they don’t work. But they work. Don't ask me why. Probably because of how OpenSSL works:

• client picks up the whole chain in a leaf cert file
• client sends the whole chain
• server validates the whole chain during handshake

Apropos! We validate certs not only during handshakes, but also explicitly. However we never consider chains, not even in CSR handling. That's why we get strange error messages like this:

[2023-06-21 16:54:42 +0000] information/JsonRpcConnection: Received certificate request for CN 'aklimov-intca-2.novalocal' not signed by our CA: unable to get local issuer certificate (code 20)

So it basically works, but it's not perfect.

Test protocol

[root@aklimov-intca-1 ~]# icinga2 feature enable api
Enabling feature api. Make sure to restart Icinga 2 for these changes to take effect.
[root@aklimov-intca-1 ~]# icinga2 daemon -C
[2023-06-21 15:47:28 +0000] information/cli: Icinga application loader (version: r2.13.7-1)
[2023-06-21 15:47:28 +0000] information/cli: Loading configuration file(s).
[2023-06-21 15:47:28 +0000] information/ConfigItem: Committing config item(s).
[2023-06-21 15:47:28 +0000] critical/SSL: Error on bio X509 AUX reading pem file '/var/lib/icinga2/certs//aklimov-intca-1.novalocal.crt': 2147483650, "error:80000002:system library::No such file or directory"
[2023-06-21 15:47:28 +0000] critical/config: Error: Cannot get certificate from cert path: '/var/lib/icinga2/certs//aklimov-intca-1.novalocal.crt'.
Location: in /etc/icinga2/features-enabled/api.conf: 5:1-5:24
/etc/icinga2/features-enabled/api.conf(3):  */
/etc/icinga2/features-enabled/api.conf(4):
/etc/icinga2/features-enabled/api.conf(5): object ApiListener "api" {
                                           ^^^^^^^^^^^^^^^^^^^^^^^^
/etc/icinga2/features-enabled/api.conf(6):   //accept_config = false
/etc/icinga2/features-enabled/api.conf(7):   //accept_commands = false
[2023-06-21 15:47:28 +0000] critical/config: 1 error
[2023-06-21 15:47:28 +0000] critical/cli: Config validation failed. Re-run with 'icinga2 daemon -C' after fixing the config.
[root@aklimov-intca-1 ~]# openssl req -x509 -newkey rsa:4096 -subj '/CN=Ext. Root CA' -md5 -keyout root.key -out root.crt -nodes
...+.............+......+........+...+...+.......+...+.....+.+..............+.+.....+......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*...+......+............+........+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*..+...+.........+.+........+......+...+...+.......+..+..........+..............+..........+......+.....+....+..+.............+.........+...........+.......+.....+..................+....+...+............+...+.....+.......+..................+..+......+.+...+..+....+......+........+...+.......+.....+.......+...+......+..+.......+........+............+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
....+.+...+.....+.........+..........+.....+.+..+...+..........+......+...+..+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*....+...+.....+...+...+............+.............+..+....+.....+...+...+....+...........+...+.+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.........+..+.+..+....+...+.....+......+.+...+...+...+.....+.+.....+...................+..+....+...+.....+.........................+...+............+.....+..........+.....+......+.......+...+....................+.........+..................+...................+...........+....+...........+.........................+..+............+.+....................+...................+.......................+......+.+...+......+......+...........+...+.+......+..................+.....................+..................+.....+.+.....+.+.....+.+.........+.....+...+..........+...+........+..................................+............+......+.....+.............+......+.....+.+.........+......+............+..+.........+......+......+....+.....+.........................+.....+.........+......+....+.....+...............+.+...........+.......+..+...+................+.....+.........+.............+..+.+............+..+.+..+...+.+........................+.....+..........+...........+......+...+...............+..........+..............+..........+..+.+.....+......................+...+..+............+..........+...........+.......+..+.+..+.......+...........+.......+..+..........+.....+.........+.......+...+...........+.........+............+..........+........+......+...+......+.+...+...........+.......+...+...........+...+..........+..+....+..+............+...+...+.+...+..+.+..+......+.+.........+.....+................+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-----
[root@aklimov-intca-1 ~]# openssl req -newkey rsa:4096 -subj '/CN=Ext. Intm. CA' -keyout intm.key -out intm.csr -nodes
....+.......+...+.....+.+.....+.+.........+...+..+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*..+...................+........+......+.+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*...+......+..+......................+.....................+..+.........+.+.........+.....+.+........+....+.....+...+..................+.........+...............+.+......+........+.+...........+....+......+.....+.........+....+..+.+..............+..................+.+..+.........+....+..............+..........+..+...............+...+......+....+...........+.+...........+.........+....+..+............+.+.........+......+.....+...............+.+...............+...............+...........+...+.......+..+...+..................+.+...+.....+......+.+......+...............+...+......+......+..+......+......+.........+......+.....................+..........+.....+...+.+.........+.....+..........+.........+...........+.+...+...+...+..+..........+..+......................+...+.....+...+...+......+...+......+.+.....................+.....+.........+....+.........+.....+.+.........+.....+...+.................................+.+......+.........+...+...+..............+...............+.+.........+........+...+....+...............+......+.............................+................+.....+........................+......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
..+...+....+......+.........+.........+...........+.+..............+....+..............+.+..+....+........+...+...+....+.....+.+..+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*...+....+........+.+.....+....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*......+.........+......+..............+...+.....................+..........+.....+.+.....+.........+.+.....................+..............+.+.....+.........+...............+...+..........+...+..+.+...........+....+...........+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-----
[root@aklimov-intca-1 ~]# openssl x509 -req -in intm.csr -sha512 -out intm.crt -CA root.crt -CAkey root.key -CAcreateserial -extensions ext -extfile <(printf '[ext]\nbasicConstraints=critical,CA:TRUE,pathlen:0')
Certificate request self-signature ok
subject=CN = Ext. Intm. CA
[root@aklimov-intca-1 ~]# openssl req -newkey rsa:4096 -subj '/CN=aklimov-intca-1.novalocal' -keyout aklimov-intca-1.novalocal.key -out aklimov-intca-1.novalocal.csr -nodes
.....+.............+.....+.........+.+...+..+.......+..+..........+.....+....+..+..................+.+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*....+...+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*..+......................+..+.......+........+.+..............................+..+..........+..+.......+...+...........+....+...+..............+.........+............+...+.......+..+...+...+.......+...+...........+...............................+..+......................+.....+....+.....+......+.+.........+...+......+........................+........+...+.+...+........+.......+..+.........+.+.........+.....+......+...+..........+...+...+.........+.................+...+...+.......+.....+.......+..............................+...+..+.........+.......+.....+..........+...+.........+.....+.+..+..........+...+..+....+...+..+.+...+...........+...+.+.....+......+.+........+....+...........+..........+..+....+.........+......+......+.....+.......+......+...........+...+.+.....+....+...........+......+.............+.....+.......+...+..+...+.....................+....+.....+.............+...+.........+...+.........+...+........+.............+..+.........+.........+.........+............+.+...+............+..+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
...+.....+.............+...+.....+.......+..+...+...+...+.+......+.....+...+.+.....+.+........+.+.........+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*...+.........+..+.+..+...+.........+.............+..+.+...+......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*....+..........+...+......+..........................+....+.....+.+...........+....+......+...+.....................+.....+.+..+.......+...+.......................+....+.....+...................+.....+.+.....+....+...........+................+...............+.....+.........+.+..+............................+...+.....+...............+...............+.+......+..............+....+..+.......+.....+.+.........+............+...........+...+.......+..+............+.+............+.....+...+....+..............+....+..+...+..........+......+......+...+...+..+......+...............+.........+...+.......+..+.+.........+...+......+...........+...............+....+..+...+............+.......+......+.....+.........+...+.......+...........+.+...+...........+......+..........+........+.+..+...+.......+..+......+......+...+............+....+.........+...+..+.......+.....+....+..+.........+...+.......+...............+........+...+............+.+......+.....+.........+....+...+..+......+......+.........+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-----
[root@aklimov-intca-1 ~]# openssl x509 -req -in aklimov-intca-1.novalocal.csr -sha512 -out aklimov-intca-1.novalocal.crt -CA intm.crt -CAkey intm.key -CAcreateserial -extensions SAN -extfile <(printf '[SAN]\nsubjectAltName=DNS:aklimov-intca-1.novalocal')
Certificate request self-signature ok
subject=CN = aklimov-intca-1.novalocal
[root@aklimov-intca-1 ~]# cp aklimov-intca-1.novalocal.* /var/lib/icinga2/certs
[root@aklimov-intca-1 ~]# cat intm.crt >> /var/lib/icinga2/certs/aklimov-intca-1.novalocal.crt
[root@aklimov-intca-1 ~]# chown icinga: /var/lib/icinga2/certs/aklimov-intca-1.novalocal.*
[root@aklimov-intca-1 ~]# icinga2 daemon -C
[2023-06-21 15:53:03 +0000] information/cli: Icinga application loader (version: r2.13.7-1)
[2023-06-21 15:53:03 +0000] information/cli: Loading configuration file(s).
[2023-06-21 15:53:03 +0000] information/ConfigItem: Committing config item(s).
[2023-06-21 15:53:03 +0000] information/ApiListener: My API identity: aklimov-intca-1.novalocal
[2023-06-21 15:53:03 +0000] critical/SSL: Error loading and verifying locations in ca key file '/var/lib/icinga2/certs//ca.crt': 2147483650, "error:80000002:system library::No such file or directory"
[2023-06-21 15:53:03 +0000] critical/config: Error: Cannot make SSL context for cert path: '/var/lib/icinga2/certs//aklimov-intca-1.novalocal.crt' key path: '/var/lib/icinga2/certs//aklimov-intca-1.novalocal.key' ca path: '/var/lib/icinga2/certs//ca.crt'.
Location: in /etc/icinga2/features-enabled/api.conf: 5:1-5:24
/etc/icinga2/features-enabled/api.conf(3):  */
/etc/icinga2/features-enabled/api.conf(4):
/etc/icinga2/features-enabled/api.conf(5): object ApiListener "api" {
                                           ^^^^^^^^^^^^^^^^^^^^^^^^
/etc/icinga2/features-enabled/api.conf(6):   //accept_config = false
/etc/icinga2/features-enabled/api.conf(7):   //accept_commands = false
[2023-06-21 15:53:03 +0000] critical/config: 1 error
[2023-06-21 15:53:03 +0000] critical/cli: Config validation failed. Re-run with 'icinga2 daemon -C' after fixing the config.
[root@aklimov-intca-1 ~]# cp root.crt /var/lib/icinga2/certs/ca.crt
[root@aklimov-intca-1 ~]# icinga2 daemon -C
[2023-06-21 15:54:06 +0000] information/cli: Icinga application loader (version: r2.13.7-1)
[2023-06-21 15:54:06 +0000] information/cli: Loading configuration file(s).
[2023-06-21 15:54:06 +0000] information/ConfigItem: Committing config item(s).
[2023-06-21 15:54:06 +0000] information/ApiListener: My API identity: aklimov-intca-1.novalocal
[2023-06-21 15:54:06 +0000] information/ConfigItem: Instantiated 12 Notifications.
[2023-06-21 15:54:06 +0000] information/ConfigItem: Instantiated 1 IcingaApplication.
[2023-06-21 15:54:06 +0000] information/ConfigItem: Instantiated 2 HostGroups.
[2023-06-21 15:54:06 +0000] information/ConfigItem: Instantiated 1 Host.
[2023-06-21 15:54:06 +0000] information/ConfigItem: Instantiated 1 FileLogger.
[2023-06-21 15:54:06 +0000] information/ConfigItem: Instantiated 1 CheckerComponent.
[2023-06-21 15:54:06 +0000] information/ConfigItem: Instantiated 3 Zones.
[2023-06-21 15:54:06 +0000] information/ConfigItem: Instantiated 1 Endpoint.
[2023-06-21 15:54:06 +0000] information/ConfigItem: Instantiated 1 ApiListener.
[2023-06-21 15:54:06 +0000] information/ConfigItem: Instantiated 1 NotificationComponent.
[2023-06-21 15:54:06 +0000] information/ConfigItem: Instantiated 244 CheckCommands.
[2023-06-21 15:54:06 +0000] information/ConfigItem: Instantiated 1 UserGroup.
[2023-06-21 15:54:06 +0000] information/ConfigItem: Instantiated 3 ServiceGroups.
[2023-06-21 15:54:06 +0000] information/ConfigItem: Instantiated 3 TimePeriods.
[2023-06-21 15:54:06 +0000] information/ConfigItem: Instantiated 1 User.
[2023-06-21 15:54:06 +0000] information/ConfigItem: Instantiated 1 ScheduledDowntime.
[2023-06-21 15:54:06 +0000] information/ConfigItem: Instantiated 11 Services.
[2023-06-21 15:54:06 +0000] information/ConfigItem: Instantiated 2 NotificationCommands.
[2023-06-21 15:54:06 +0000] information/ScriptGlobal: Dumping variables to file '/var/cache/icinga2/icinga2.vars'
[2023-06-21 15:54:06 +0000] information/cli: Finished validating the configuration file(s).
[root@aklimov-intca-1 ~]#

[root@aklimov-intca-2 ~]# openssl req -newkey rsa:4096 -subj '/CN=aklimov-intca-2.novalocal' -keyout aklimov-intca-2.novalocal.key -out aklimov-intca-2.novalocal.csr -nodes
...........+.....+...+......+..........+.....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*..+....+.........+..+..........+.....+...+......+.........+.......+..+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*....+....+...........+..........+..............+...+.......+...+...........+........................+............+.............+...+.....+...+......+...+......+............+....+...+...+..+................+.................+.......+..+....+......+........+.+..+.........+...+.......+.....+....+...........+......+..........+...+...........+......+...........................+..........+..+..........+........+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
.......+...+.+.........+..+...+....+..+.+........+.........+...+...+....+............+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*...+........+....+...............+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*....+...........+...+.......+........+....+...+..+...............+.+...........+.........+......+......+.............+.................+...+............+.......+.....+.......+...............+...........+.......+...+...............+..+.+..+....+.....+...+.+......+........+.+..+....+..............+.+.....+..........+...+...+........+............+...+......+...................+...............+.........+..+.+..+......+.....................+.+.....+.......+...+...+.....+.........+..................+..........+........+.+........+............+.....................+..................+....+.....+..........+..+...+.+......+......+.........+...+..+.....................+....+...+...........+......+.........+....+........+.............+..+....+...+........+.........+.+........+......+...+..................+...+.+.....+......+..........+..+...+....+........+......+.+...+.....+.......+.....+..................+.+.....+......+................+...............+............+............+......+.....+....+...............+....................+................+.....+.........+.+.........+...............+.........+..+..........+...+.....+..........+..+..........+.........+...+..+.+........+.........+..........+......+..+.......+...........+......+.........+.+.....+.+.....+..........+..+.......+.........+..............................+............+...+...........+.+...+............+..+.............+..+...+....+.........+.........+......+..+....+.....+......+............+.............+...............+........+....+..........................+......+......+............+...+....+........+...+....+.........+......+.........+......+......+..+...+...+....+......+...............+...............+...............+..+...+..........+.....+.+.....+.........+.+.....+.+..+.............+........+.........+.+......+........+...............+...+............+.............+..+...+..............................+.......+..+.+..+.+......+...+......+..+..........+..+.+......+.....+.+........+.+...........+...+......+...+......+.+.....................+..+...+....+.....+......+.............+......+.....+.+..+......+.......+..............+......+..........+.....+.......+..+...+.....................+..........+.........+............+...........+.......+..................+............+..+....+..............+......+.....................+.+....................+.........+..........+..................+...+.....+......................+...+..+...+...+......+..........+.........+.....+......+.............+..+.+.................+.+.........+........+.............+......+...+............+........+.......+...+.....+......+......+....+..+.........+..................+..........+...+..+...............+......+................+...........+....+...+......+.........+..+...+..........+............+.........+..+...+.+...+..+..........+.....+....+............+..............+..................................+...............+...........+..........+.....+....+.........+...+.....+....+......+...+.....+....+...+...+...............+...............+..+.........+.........+......+.+.....+...+.+.....+...+..........+..+.+.....+...+....+......+......+...+.....+...+...+.+......+.....+............+.......+.........+..+.......+..+..........+...+...........+....+.................+.........+.+........+.............+..+............+...+......+....+..+.+............+..+......+...+....+...+...+..+...............+..........+...+..+....+...+...........................+......+........+.+...........+.......+...+.....+..................+...............+.+...+.....+......+.+........+...+.......+........+..........+........+............+.......+..+....+......+...+......+......+.....+...............+.+.........+.....+...+...+....+.........+.................+.............+......+..............+.+..............+.+.....+.+.......................+......+................+...+..+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-----
[root@aklimov-intca-2 ~]#

(copies aklimov-intca-2.novalocal.csr over)

[root@aklimov-intca-1 ~]# openssl x509 -req -in aklimov-intca-2.novalocal.csr -sha512 -out aklimov-intca-2.novalocal.crt -CA intm.crt -CAkey intm.key -CAcreateserial -extensions SAN -extfile <(printf '[SAN]\nsubjectAltName=DNS:aklimov-intca-2.novalocal')
Certificate request self-signature ok
subject=CN = aklimov-intca-2.novalocal
[root@aklimov-intca-1 ~]# cat intm.crt >> aklimov-intca-2.novalocal.crt
[root@aklimov-intca-1 ~]#

(copies aklimov-intca-2.novalocal.crt and root.crt over)

[root@aklimov-intca-2 ~]# cp aklimov-intca-2.novalocal.* /var/lib/icinga2/certs
[root@aklimov-intca-2 ~]# chown icinga: /var/lib/icinga2/certs/aklimov-intca-2.novalocal.*
[root@aklimov-intca-2 ~]# cp root.crt /var/lib/icinga2/certs/ca.crt
[root@aklimov-intca-2 ~]# icinga2 feature enable api
Enabling feature api. Make sure to restart Icinga 2 for these changes to take effect.
[root@aklimov-intca-2 ~]# icinga2 daemon -C
[2023-06-21 16:17:32 +0000] information/cli: Icinga application loader (version: r2.13.7-1)
[2023-06-21 16:17:32 +0000] information/cli: Loading configuration file(s).
[2023-06-21 16:17:32 +0000] information/ConfigItem: Committing config item(s).
[2023-06-21 16:17:32 +0000] information/ApiListener: My API identity: aklimov-intca-2.novalocal
[2023-06-21 16:17:32 +0000] information/ConfigItem: Instantiated 12 Notifications.
[2023-06-21 16:17:32 +0000] information/ConfigItem: Instantiated 1 IcingaApplication.
[2023-06-21 16:17:32 +0000] information/ConfigItem: Instantiated 2 HostGroups.
[2023-06-21 16:17:32 +0000] information/ConfigItem: Instantiated 1 Host.
[2023-06-21 16:17:32 +0000] information/ConfigItem: Instantiated 1 FileLogger.
[2023-06-21 16:17:32 +0000] information/ConfigItem: Instantiated 1 CheckerComponent.
[2023-06-21 16:17:32 +0000] information/ConfigItem: Instantiated 3 Zones.
[2023-06-21 16:17:32 +0000] information/ConfigItem: Instantiated 1 Endpoint.
[2023-06-21 16:17:32 +0000] information/ConfigItem: Instantiated 1 ApiListener.
[2023-06-21 16:17:32 +0000] information/ConfigItem: Instantiated 1 NotificationComponent.
[2023-06-21 16:17:32 +0000] information/ConfigItem: Instantiated 244 CheckCommands.
[2023-06-21 16:17:32 +0000] information/ConfigItem: Instantiated 1 UserGroup.
[2023-06-21 16:17:32 +0000] information/ConfigItem: Instantiated 3 ServiceGroups.
[2023-06-21 16:17:32 +0000] information/ConfigItem: Instantiated 3 TimePeriods.
[2023-06-21 16:17:32 +0000] information/ConfigItem: Instantiated 1 User.
[2023-06-21 16:17:32 +0000] information/ConfigItem: Instantiated 1 ScheduledDowntime.
[2023-06-21 16:17:32 +0000] information/ConfigItem: Instantiated 11 Services.
[2023-06-21 16:17:32 +0000] information/ConfigItem: Instantiated 2 NotificationCommands.
[2023-06-21 16:17:33 +0000] information/ScriptGlobal: Dumping variables to file '/var/cache/icinga2/icinga2.vars'
[2023-06-21 16:17:33 +0000] information/cli: Finished validating the configuration file(s).
[root@aklimov-intca-2 ~]# cat << EOF > /etc/icinga2/zones.conf
object Endpoint "aklimov-intca-1.novalocal" {
  host = "10.27.0.143"
}
object Endpoint "aklimov-intca-2.novalocal" {
  host = "10.27.0.184"
}
object Zone "master" {
  endpoints = [ "aklimov-intca-1.novalocal", "aklimov-intca-2.novalocal" ]
}
EOF
[root@aklimov-intca-2 ~]#

[root@aklimov-intca-1 ~]# cat << EOF > /etc/icinga2/zones.conf
object Endpoint "aklimov-intca-1.novalocal" {
  host = "10.27.0.143"
}
object Endpoint "aklimov-intca-2.novalocal" {
  host = "10.27.0.184"
}
object Zone "master" {
  endpoints = [ "aklimov-intca-1.novalocal", "aklimov-intca-2.novalocal" ]
}
EOF
[root@aklimov-intca-1 ~]#

Now, if I fire up the Icingas, they recognise each other. With config acceptance enabled in the API feature they even sync config to each other.

[root@aklimov-intca-2 ~]# cat /var/lib/icinga2/api/zones/master/_etc/my.conf
object Host "demo.kli.mov" {
	check_command = "dummy"
}
[root@aklimov-intca-2 ~]#

And... now what?

@sircubbi, you have already made PRs in this topic and seem to understand it IMAO. Depending on what you need, please do any of the following. The headings are independent of each other, but the bullets per heading should be done in their order. At best even one after the other's merged, to save worktime. Finally (or even before) indicate what you won’t (need and) do.

Intermediate CAs

• Add the fact that (and how) external intermediate CAs already work (see above) to the Icinga docs (force push #8859)
  • I did in #9825
• Continue with PRs like #9795 with focus on nicely handling fully external intermediate CAs (no CA keys in Icinga dirs)
• In a separate PR make Icinga-owned intermediate CA (see above) working – but I doubt you'll need/do this as int. CAs are likely to be fully external
• If you're really bored after all of the above, feel free to evaluate whether Icinga could do https://github.com/Icinga/icinga2/issues/7719#issuecomment-1551090384 automagically (i.e. int. CA used as root CA, Icinga auto-cross-self-signs it in memory w/o its private key)

Multiple roots

A little off-topic, but you've already done something on it: https://github.com/Icinga/icinga2/pull/8859#issuecomment-1599115007

• Add the fact that (and how) multiple root CAs already work (see below) to the Icinga docs
  • I did in #9825
• Test how well multiple root CAs (no intermediates!) already work in /var/lib/icinga2/certs/ca.crt. Either report success or make a new PR.

Start with the certificate paths and API configuration described in the issue, then reproduce the intermediate-CA scenarios using the shown OpenSSL commands and icinga2 daemon -C. Trace the certificate and CSR validation behavior for complete chains; done means Icinga-owned and external intermediate CAs are handled without the reported verification errors.