Invalid Certificate chain when changing Connect CA from Consul to Vault if Cross-Signing is enabled.
- Dominant language
- Go
- Stars
- 30.1k
- Forks
- 4.6k
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 43
Description
#### Overview of the Issue
When changing Consul Connect CA from Consul to Vault, an invalid Certificate chain is generated if cross-signing is enabled.
---
#### Reproduction Steps
- Standard Consul Cluster with TLS and ACLs enabled
- External Root CA with a valid intermediate CA Cert issued to Vault to act as the Root CA for Consul (loaded into Vault with the intemediate and root cert bundle as per https://developer.hashicorp.com/consul/docs/connect/ca/vault#rootpkipath)
- Follow https://developer.hashicorp.com/consul/tutorials/vault-secure/vault-pki-consul-connect-ca using an existing consul cluster
- New settings will deploy correctly and consul will issue certificate
- When loading the UI in a web browser to view it, the Cert will show as invalid and is missing part of the CA Chain to verify validity.
However if you change the CA to vault while using "ForceWithoutCrossSigning" set to True, it will correctly issue certs with the full cert chain.
possibly related to #16592
-->
### Consul info for both Client and Server
Client info
```
agent:
check_monitors = 0
check_ttls = 0
checks = 2
services = 3
build:
prerelease =
revision = 5e08e229
version = 1.15.2
version_metadata =
consul:
acl = enabled
known_servers = 3
server = false
runtime:
arch = amd64
cpu_count = 1
goroutines = 69
max_procs = 1
os = linux
version = go1.20.1
serf_lan:
coordinate_resets = 0
encrypted = true
event_queue = 0
event_time = 25
failed = 0
health_score = 0
intent_queue = 0
left = 0
member_time = 80185
members = 15
query_queue = 0
query_time = 1
```
```
auto_config {
enabled = true
intro_token_file = "/etc/consul.d/jwt-token"
server_addresses = ["provider=vsphere category_name=terraform-managed tag_name=terraform-consul-server host=vcsa.ONPREMLAB.local user=administrator@ONPREMLAB.local password=REDACTED"]
}
tls {
defaults {
verify_incoming = false
verify_outgoing = true
ca_file = "/etc/consul.d/certs/consul-agent-ca.pem"
}
internal_rpc {
verify_server_hostname = true
}
}
bind_addr = "0.0.0.0"
advertise_addr = "{{ GetPrivateInterfaces | include \"network\" \"192.168.253.0/24\" | attr \"address\" }}"
datacenter = "ONPREMLAB"
data_dir = "/opt/consul"
retry_join = ["provider=vsphere category_name=terraform-managed tag_name=terraform-consul-server host=vcsa.ONPREMLAB.local user=administrator@ONPREMLAB.local password=REDACTED"]
ports {
http = -1
https = 8501
grpc = -1
grpc_tls = 8503
}
acl {
enabled = true
default_policy = "deny"
enable_token_persistence = true
}
```
Server info
```
agent:
check_monitors = 0
check_ttls = 0
checks = 0
services = 0
build:
prerelease =
revision = 5e08e229
version = 1.15.2
version_metadata =
consul:
acl = enabled
bootstrap = false
known_datacenters = 1
leader = false
leader_addr = 192.168.253.18:8300
server = true
raft:
applied_index = 628244
commit_index = 628244
fsm_pending = 0
last_contact = 361.803µs
last_log_index = 628244
last_log_term = 114064
last_snapshot_index = 622786
last_snapshot_term = 114063
latest_configuration = [....]
latest_configuration_index = 0
num_peers = 2
protocol_version = 3
protocol_version_max = 3
protocol_version_min = 0
snapshot_version_max = 1
snapshot_version_min = 0
state = Follower
term = 114064
runtime:
arch = amd64
cpu_count = 2
goroutines = 167
max_procs = 2
os = linux
version = go1.20.1
serf_lan:
coordinate_resets = 0
encrypted = true
event_queue = 0
event_time = 25
failed = 0
health_score = 0
intent_queue = 0
left = 0
member_time = 80185
members = 15
query_queue = 0
query_time = 1
serf_wan:
coordinate_resets = 0
encrypted = true
event_queue = 0
event_time = 1
failed = 0
health_score = 0
intent_queue = 0
left = 0
member_time = 214
members = 3
query_queue = 0
query_time = 1
```
```
datacenter = "ONPREMLAB"
data_dir = "/opt/consul"
encrypt = "REDACTED"
ports {
http = -1
https = 8501
grpc = -1
grpc_tls = 8503
}
acl {
enabled = true
default_policy = "deny"
enable_token_persistence = true
}
tls {
defaults {
verify_incoming = true
verify_outgoing = true
ca_file = "/etc/consul.d/certs/consul-agent-ca.pem"
cert_file = "/etc/consul.d/certs/ONPREMLAB-server-consul-0.pem"
key_file = "/etc/consul.d/certs/ONPREMLAB-server-consul-0-key.pem"
}
internal_rpc {
verify_server_hostname = true
}
}
auto_config {
authorization {
enabled = true
static {
oidc_discovery_url = "https://vaultcluster.ONPREMLAB.local:8200/v1/identity/oidc"
bound_issuer = "https://vaultcluster.ONPREMLAB.local:8200/v1/identity/oidc"
bound_audiences = ["consul-cluster-ONPREMLAB"]
}
}
}
server = true
bootstrap_expect = 3
connect {
enabled = true
}
addresses {
dns = "0.0.0.0"
https = "127.0.0.1"
grpc_tls = "127.0.0.1"
}
ui_config {
enabled = true
}
advertise_addr = "{{ GetPrivateInterfaces | include \"network\" \"192.168.253.0/24\" | attr \"address\" }}"
telemetry {
prometheus_retention_time = "60s"
disable_hostname = true
}
```
### Operating system and Environment details
OS: Ubuntu 20.04.6 LTS (GNU/Linux 5.4.0-148-generic x86_64)
### Snippets
swap CA commands:
consul connect ca set-config -config-file config-connect-ca-provider-vault.json
consul connect ca set-config -config-file config-connect-ca-provider-consul.json
config-connect-ca-provider-consul.json
{
"Provider": "consul",
"Config": {
"IntermediateCertTTL": "8760h",
"LeafCertTTL": "72h",
"RootCertTTL": "87600h"
},
"State": null,
"ForceWithoutCrossSigning": false,
"CreateIndex": 8,
"ModifyIndex": 12
}
config-connect-ca-provider-vault.json
{
"Provider": "vault",
"Config": {
"Address": "https://vaultcluster.ONPREMLAB.local:8200",
"Token": "REDACTED",
"RootPKIPath": "consul_connect_root_ca",
"IntermediatePKIPath": "consul_connect_ca",
"LeafCertTTL": "72h",
"RotationPeriod": "2160h",
"IntermediateCertTTL": "8760h",
"PrivateKeyType": "rsa",
"PrivateKeyBits": 2048
},
"ForceWithoutCrossSigning": true
}
Contributor guide
Assessment
This issue has not been assessed yet.