hashicorp / hashicorp/consul-template

Commands are not aggregating when running multiple templates

Open
#2,030 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
4.8k
Forks
801
Avg merge
4h 5m
Merged PRs (30d)
6

Description

### Configuration

```hcl
# {{ ansible_managed }}

vault {
address = "http://127.0.0.1:8100"
renew_token = false
retry {
# Settings to 0 for unlimited retries.
attempts = 0
}
}

consul {
address = "127.0.0.1:8500"
retry {
# Settings to 0 for unlimited retries.
attempts = 0
}
}

wait {
min = "15s"
max = "30s"
}

# Postgres CA
template {
source = "/etc/consul-template/templates/keystone/postgres-server-ca.crt.ctmpl"
destination = "/etc/keystone/postgres-server-ca.crt"
create_dest_dirs = false
perms = "0644"
exec {
command = "sudo systemctl reload-or-restart apache2 || true"
}
}

# Postgres User
template {
source = "/etc/consul-template/templates/keystone/postgres-user-keystone.ctmpl"
destination = "/etc/keystone/postgres-user-keystone.rendered"
create_dest_dirs = false
perms = "0600"
exec {
command = "sudo systemctl reload-or-restart apache2 || true"
}
}
```

`/etc/consul-template/templates/keystone/postgres-server-ca.crt.ctmpl`
```liquid
{{ with secret "pki_openstack_postgres_root/cert/ca" }}
{{ .Data.certificate }}
{{ end }}
```

`/etc/consul-template/templates/keystone/postgres-user-keystone.ctmpl`
```liquid
{{- with pkiCert "pki_openstack_postgres_intermediate/issue/user-keystone" "common_name=keystone" -}}
{{ .Cert }}{{ .CA }}{{ .CAChain }}{{ .Key }}
{{ .Key | writeToFile "/etc/keystone/postgres-user-keystone.key" "" "" "0600" }}
{{ (printf "%s%s" .Cert (.CAChain | join "")) | writeToFile "/etc/keystone/postgres-user-keystone.crt" "" "" "0644" }}
{{- end -}}
```

### Command

```shell
/usr/bin/consul-template -config=/etc/consul-template/consul-template-keystone.hcl -log-level=debug
```

### Debug output

```
Mar 16 13:46:14 openstack-keystone-1 systemd[1]: Starting consul-template-keystone.service - "consul-template-keystone"...
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.389-0500 [INFO] consul-template v0.40.0 (781ce19)
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.389-0500 [INFO] (runner) creating new runner (dry: false, once: false)
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.389-0500 [DEBUG] (runner) final config: {"Consul":{"Address":"127.0.0.1:8500","Namespace":"","Auth":{"Enabled":false,"Username":""},"Retry":{"Attempts":0,"Backoff":250000000,"MaxBackoff":60000000000,"E
nabled":true},"SSL":{"CaCert":"","CaCertBytes":"","CaPath":"","Cert":"","Enabled":false,"Key":"","ServerName":"","Verify":true},"Token":"","TokenFile":"","Transport":{"CustomDialer":null,"DialKeepAlive":30000000000,"DialTimeout":30000000000,"DisableKeepAlives":false,"IdleConnTimeo
ut":5000000000,"MaxIdleConns":0,"MaxIdleConnsPerHost":100,"MaxConnsPerHost":0,"TLSHandshakeTimeout":10000000000}},"Dedup":{"Enabled":false,"MaxStale":2000000000,"Prefix":"consul-template/dedup/","TTL":15000000000,"BlockQueryWaitTime":60000000000},"DefaultDelims":{"Left":null,"Righ
t":null},"Exec":{"Command":[],"Enabled":false,"Env":{"Denylist":[],"Custom":[],"Pristine":false,"Allowlist":[]},"KillSignal":2,"KillTimeout":30000000000,"ReloadSignal":null,"Splay":0,"Timeout":0},"KillSignal":2,"LogLevel":"debug","FileLog":{"LogFilePath":"","LogRotateBytes":0,"Log
RotateDuration":86400000000000,"LogRotateMaxFiles":0},"MaxStale":2000000000,"PidFile":"","ReloadSignal":1,"Syslog":{"Enabled":false,"Facility":"LOCAL0","Name":"consul-template"},"Templates":[{"Backup":false,"Command":[],"CommandTimeout":30000000000,"Contents":"","CreateDestDirs":f
alse,"Destination":"/etc/keystone/postgres-server-ca.crt","ErrMissingKey":false,"ErrFatal":true,"Exec":{"Command":["sudo systemctl reload-or-restart apache2 || true"],"Enabled":true,"Env":{"Denylist":[],"Custom":[],"Pristine":false,"Allowlist":[]},"KillSignal":2,"KillTimeout":3000
0000000,"ReloadSignal":null,"Splay":0,"Timeout":30000000000},"Perms":420,"User":null,"Uid":null,"Group":null,"Gid":null,"Source":"/etc/consul-template/templates/keystone/postgres-server-ca.crt.ctmpl","Wait":{"Enabled":false,"Min":0,"Max":0},"LeftDelim":"","RightDelim":"","Function
Denylist":[],"SandboxPath":"","MapToEnvironmentVariable":""},{"Backup":false,"Command":[],"CommandTimeout":30000000000,"Contents":"","CreateDestDirs":false,"Destination":"/etc/keystone/postgres-user-keystone.rendered","ErrMissingKey":false,"ErrFatal":true,"Exec":{"Command":["sudo
systemctl reload-or-restart apache2 || true"],"Enabled":true,"Env":{"Denylist":[],"Custom":[],"Pristine":false,"Allowlist":[]},"KillSignal":2,"KillTimeout":30000000000,"ReloadSignal":null,"Splay":0,"Timeout":30000000000},"Perms":384,"User":null,"Uid":null,"Group":null,"Gid":null,"
Source":"/etc/consul-template/templates/keystone/postgres-user-keystone.ctmpl","Wait":{"Enabled":false,"Min":0,"Max":0},"LeftDelim":"","RightDelim":"","FunctionDenylist":[],"SandboxPath":"","MapToEnvironmentVariable":""}],"TemplateErrFatal":null,"Vault":{"Address":"http://127.0.0.
1:8100","Enabled":true,"Namespace":"","RenewToken":false,"Retry":{"Attempts":0,"Backoff":250000000,"MaxBackoff":60000000000,"Enabled":true},"SSL":{"CaCert":"","CaCertBytes":"","CaPath":"","Cert":"","Enabled":true,"Key":"","ServerName":"","Verify":true},"Transport":{"CustomDialer":null,"DialKeepAlive":30000000000,"DialTimeout":30000000000,"DisableKeepAlives":false,"IdleConnTimeout":5000000000,"MaxIdleConns":0,"MaxIdleConnsPerHost":100,"MaxConnsPerHost":0,"TLSHandshakeTimeout":10000000000},"UnwrapToken":false,"ClientUserAgent":null,"DefaultLeaseDuration":300000000000,"LeaseRenewalThreshold":0.9,"K8SAuthRoleName":"","K8SServiceAccountTokenPath":"/run/secrets/kubernetes.io/serviceaccount/token","K8SServiceAccountToken":"","K8SServiceMountPath":"kubernetes"},"Nomad":{"Address":"","Enabled":false,"Namespace":"","SSL":{"CaCert":"","CaCertBytes":"","CaPath":"","Cert":"","Enabled":false,"Key":"","ServerName":"","Verify":true},"AuthUsername":"","AuthPassword":"","Transport":{"CustomDialer":null,"DialKeepAlive":30000000000,"DialTimeout":30000000000,"DisableKeepAlives":false,"IdleConnTimeout":5000000000,"MaxIdleConns":0,"MaxIdleConnsPerHost":100,"MaxConnsPerHost":0,"TLSHandshakeTimeout":10000000000},"Retry":{"Attempts":12,"Backoff":250000000,"MaxBackoff":60000000000,"Enabled":true}},"Wait":{"Enabled":true,"Min":15000000000,"Max":30000000000},"Once":false,"ParseOnly":false,"BlockQueryWaitTime":60000000000,"ErrOnFailedLookup":false}
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.390-0500 [INFO] (runner) creating watcher
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.390-0500 [INFO] (runner) starting
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.390-0500 [DEBUG] (runner) running initial templates
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.390-0500 [DEBUG] (runner) initiating run
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.390-0500 [DEBUG] (runner) checking template 006f8d64e84387e3303551d1250a1317
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.390-0500 [DEBUG] (runner) missing data for 1 dependencies
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.390-0500 [DEBUG] (runner) missing dependency: vault.read(pki_openstack_postgres_root/cert/ca)
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.390-0500 [DEBUG] (runner) add used dependency vault.read(pki_openstack_postgres_root/cert/ca) to missing since isLeader but do not have a watcher
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.390-0500 [DEBUG] (runner) was not watching 1 dependencies
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.390-0500 [DEBUG] (watcher) adding vault.read(pki_openstack_postgres_root/cert/ca)
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.390-0500 [DEBUG] (runner) checking template 9286f60654e88cdcc36d3f04e47ccdc9
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.390-0500 [DEBUG] (runner) missing data for 1 dependencies
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.390-0500 [DEBUG] (runner) missing dependency: vault.pki(pki_openstack_postgres_intermediate/issue/user-keystone->/etc/keystone/postgres-user-keystone.rendered)
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.390-0500 [DEBUG] (runner) add used dependency vault.pki(pki_openstack_postgres_intermediate/issue/user-keystone->/etc/keystone/postgres-user-keystone.rendered) to missing since isLeader but do not have a watcher
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.390-0500 [DEBUG] (runner) was not watching 1 dependencies
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.390-0500 [DEBUG] (watcher) adding vault.pki(pki_openstack_postgres_intermediate/issue/user-keystone->/etc/keystone/postgres-user-keystone.rendered)
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.390-0500 [DEBUG] (runner) diffing and updating dependencies
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.390-0500 [DEBUG] (runner) watching 2 dependencies
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.416-0500 [DEBUG] (runner) receiving dependency vault.pki(pki_openstack_postgres_intermediate/issue/user-keystone->/etc/keystone/postgres-user-keystone.rendered)
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.416-0500 [DEBUG] (runner) initiating run
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.416-0500 [DEBUG] (runner) checking template 006f8d64e84387e3303551d1250a1317
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.417-0500 [DEBUG] (runner) missing data for 1 dependencies
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.417-0500 [DEBUG] (runner) missing dependency: vault.read(pki_openstack_postgres_root/cert/ca)
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.417-0500 [DEBUG] (runner) missing data for 1 dependencies
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.417-0500 [DEBUG] (runner) checking template 9286f60654e88cdcc36d3f04e47ccdc9
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.417-0500 [DEBUG] (runner) rendering "/etc/consul-template/templates/keystone/postgres-user-keystone.ctmpl" => "/etc/keystone/postgres-user-keystone.rendered"
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.478-0500 [INFO] (runner) rendered "/etc/consul-template/templates/keystone/postgres-user-keystone.ctmpl" => "/etc/keystone/postgres-user-keystone.rendered"
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.478-0500 [DEBUG] (runner) appending command ["sudo systemctl reload-or-restart apache2 || true"] from "/etc/consul-template/templates/keystone/postgres-user-keystone.ctmpl" => "/etc/keystone/postgres-user-keystone.rendered"
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.478-0500 [DEBUG] (runner) diffing and updating dependencies
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.478-0500 [DEBUG] (runner) vault.read(pki_openstack_postgres_root/cert/ca) is still needed
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.478-0500 [DEBUG] (runner) vault.pki(pki_openstack_postgres_intermediate/issue/user-keystone->/etc/keystone/postgres-user-keystone.rendered) is still needed
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.478-0500 [INFO] (runner) executing command "[\"sudo systemctl reload-or-restart apache2 || true\"]" from "/etc/consul-template/templates/keystone/postgres-user-keystone.ctmpl" => "/etc/keystone/postgres-user-keystone.rendered"
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.478-0500 [INFO] (child) spawning: sh -c sudo systemctl reload-or-restart apache2 || true
Mar 16 13:46:14 openstack-keystone-1 sudo[6379]: keystone : PWD=/ ; USER=root ; COMMAND=/usr/bin/systemctl reload-or-restart apache2
Mar 16 13:46:14 openstack-keystone-1 sudo[6379]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=110)
Mar 16 13:46:14 openstack-keystone-1 sudo[6379]: pam_unix(sudo:session): session closed for user root
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.537-0500 [DEBUG] (runner) watching 2 dependencies
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.537-0500 [DEBUG] (runner) receiving dependency vault.read(pki_openstack_postgres_root/cert/ca)
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.537-0500 [DEBUG] (runner) initiating run
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.537-0500 [DEBUG] (runner) checking template 006f8d64e84387e3303551d1250a1317
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.537-0500 [DEBUG] (runner) rendering "/etc/consul-template/templates/keystone/postgres-server-ca.crt.ctmpl" => "/etc/keystone/postgres-server-ca.crt"
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.555-0500 [INFO] (runner) rendered "/etc/consul-template/templates/keystone/postgres-server-ca.crt.ctmpl" => "/etc/keystone/postgres-server-ca.crt"
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.555-0500 [DEBUG] (runner) appending command ["sudo systemctl reload-or-restart apache2 || true"] from "/etc/consul-template/templates/keystone/postgres-server-ca.crt.ctmpl" => "/etc/keystone/postgres-server-ca.crt"
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.555-0500 [DEBUG] (runner) checking template 9286f60654e88cdcc36d3f04e47ccdc9
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.555-0500 [DEBUG] (runner) rendering "/etc/consul-template/templates/keystone/postgres-user-keystone.ctmpl" => "/etc/keystone/postgres-user-keystone.rendered"
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.555-0500 [DEBUG] (runner) diffing and updating dependencies
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.555-0500 [DEBUG] (runner) vault.read(pki_openstack_postgres_root/cert/ca) is still needed
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.555-0500 [DEBUG] (runner) vault.pki(pki_openstack_postgres_intermediate/issue/user-keystone->/etc/keystone/postgres-user-keystone.rendered) is still needed
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.555-0500 [INFO] (runner) executing command "[\"sudo systemctl reload-or-restart apache2 || true\"]" from "/etc/consul-template/templates/keystone/postgres-server-ca.crt.ctmpl" => "/etc/keystone/postgres-server-ca.crt"
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.555-0500 [INFO] (child) spawning: sh -c sudo systemctl reload-or-restart apache2 || true
Mar 16 13:46:14 openstack-keystone-1 sudo[6393]: keystone : PWD=/ ; USER=root ; COMMAND=/usr/bin/systemctl reload-or-restart apache2
Mar 16 13:46:14 openstack-keystone-1 sudo[6393]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=110)
Mar 16 13:46:14 openstack-keystone-1 sudo[6393]: pam_unix(sudo:session): session closed for user root
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.721-0500 [DEBUG] (runner) watching 2 dependencies
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.721-0500 [DEBUG] (runner) all templates rendered
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.721-0500 [DEBUG] (runner) enabling global quiescence for "006f8d64e84387e3303551d1250a1317"
Mar 16 13:46:14 openstack-keystone-1 consul-template[6372]: 2025-03-16T13:46:14.721-0500 [DEBUG] (runner) enabling global quiescence for "9286f60654e88cdcc36d3f04e47ccdc9"
```

### Expected behavior

The command `sudo systemctl reload-or-restart apache2 || true` to only run once on initial rendering.

### Actual behavior

What actually happened?

The command `sudo systemctl reload-or-restart apache2 || true` ran multiple times.

You can see it ran on `2025-03-16T13:46:14.478-0500` and `2025-03-16T13:46:14.555-0500`

### Steps to reproduce

1. Use my config and templates above.

Contributor guide

Open the contributing guide

Research direction

Reproduce the behavior with the supplied Consul Template configuration and debug output, focusing on how exec commands from multiple templates are collected and run. Confirm the expected aggregation behavior when both templates render, then verify that the resulting command handling matches that behavior without breaking single-template execution.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.