hashicorp / hashicorp/consul-template
Vault KVs are not dynamically rendered by Consul-Template
- Dominant language
- Go
- Stars
- 4.8k
- Forks
- 801
- Avg merge
- 4h 5m
- Merged PRs (30d)
- 6
Description
### Consul Template version
v0.26.0 (3b7f233a)
also I'm tried to use https://github.com/ekristen/consul-template-plugin-savetofile plugin
### Configuration
```hcl
# Copy-paste your configuration files here. Only include what is necessary or
# what you've changed from defaults. Include all referenced configurations.
vault {
address = "https://"
token =
unwrap_token = false
renew_token = true
retry {
enabled = true
attempts = 60
backoff = "300ms"
max_backoff = "1m"
}
ssl {
enabled = false
}
}
syslog {
enabled = true
facility = "LOCAL5"
}
deduplicate {
enabled = false
}
```
```liquid
# Copy-paste your Consul Template template here
```
template {
destination = "/local/domain_list.txt"
create_dest_dirs = true
contents = </%s" .) }}{{ range $k, $v := .Data }}
{{ if eq $k "ca.cert" }}
{{ $v | plugin $savetofile "create" (print $path $domain "_ecc/" $k) "1000" "1000" "775" }}
{{ else }}
{{ if eq $k "crt.cert" }}
{{ $v | plugin $savetofile "create" (print $path $domain "_ecc/" $k) "1000" "1000" "775" }}
{{ else }}
{{ if eq $k "priv.key" }}
{{ $v | plugin $savetofile "create" (print $path $domain "_ecc/" $k) "1000" "1000" "775" }}
{{ else }}
{{ if eq $k "fullchain" }}
{{ $v | plugin $savetofile "create" (print $path $domain "_ecc/" $k) "1000" "1000" "755" }}
{{ end }}{{ end }}{{ end }}{{ end }}{{ end }}{{ end }}{{ end }}{{ end }}
EOH
error_on_missing_key = false
perms = 0775
backup = true
wait {
min = "2s"
max = "30s"
}
}
exec {
command = "/bin/systemctl reload nginx"
}
```liquid
# Include sample data you reference in the template from Consul or Vault here.
```
### Debug output
2021-08-11T11:16:15.223+0300 [DEBUG] (cli) receiving signal "child exited"
2021-08-11T11:16:15.224+0300 [DEBUG] (cli) receiving signal "child exited"
2021-08-11T11:16:15.225+0300 [DEBUG] (cli) receiving signal "child exited"
2021-08-11T11:16:15.227+0300 [DEBUG] (cli) receiving signal "child exited"
2021-08-11T11:16:15.228+0300 [DEBUG] (cli) receiving signal "child exited"
2021-08-11T11:16:15.229+0300 [DEBUG] (cli) receiving signal "child exited"
2021-08-11T11:16:15.230+0300 [DEBUG] (cli) receiving signal "child exited"
2021-08-11T11:16:15.231+0300 [DEBUG] (cli) receiving signal "urgent I/O condition"
2021-08-11T11:16:15.232+0300 [DEBUG] (cli) receiving signal "child exited"
2021-08-11T11:16:15.233+0300 [DEBUG] (cli) receiving signal "child exited"
2021-08-11T11:16:15.234+0300 [DEBUG] (cli) receiving signal "child exited"
2021-08-11T11:16:15.235+0300 [DEBUG] (cli) receiving signal "child exited"
2021-08-11T11:16:15.237+0300 [DEBUG] (cli) receiving signal "child exited"
2021-08-11T11:16:15.238+0300 [DEBUG] (cli) receiving signal "child exited"
2021-08-11T11:16:15.239+0300 [DEBUG] (runner) rendering "(dynamic)" => "/local/adtech/certbot/live/domain_list.txt"
2021-08-11T11:16:15.239+0300 [DEBUG] (runner) diffing and updating dependencies
2021-08-11T11:16:15.239+0300 [DEBUG] (runner) vault.read(secret//) is still needed
2021-08-11T11:16:15.239+0300 [DEBUG] (runner) vault.list(secret//) is still needed
2021-08-11T11:16:15.239+0300 [DEBUG] (runner) vault.read(secret//) is still needed
2021-08-11T11:16:15.239+0300 [DEBUG] (runner) vault.read(secret//) is still needed
2021-08-11T11:16:15.239+0300 [DEBUG] (runner) vault.read(secret//) is still needed
2021-08-11T11:16:15.239+0300 [DEBUG] (runner) vault.read(secret//) is still needed
2021-08-11T11:16:15.239+0300 [DEBUG] (runner) vault.read(secret//) is still needed
2021-08-11T11:16:15.239+0300 [DEBUG] (runner) vault.read(secret//) is still needed
.........
2021-08-11T11:16:15.239+0300 [DEBUG] (runner) vault.read(secret//) is still needed
2021-08-11T11:16:15.239+0300 [DEBUG] (runner) vault.read(secret//) is still needed
2021-08-11T11:16:15.239+0300 [DEBUG] (runner) watching 21 dependencies
2021-08-11T11:16:15.239+0300 [DEBUG] (runner) all templates rendered
2021-08-11T11:16:15.240+0300 [DEBUG] (runner) enabling template-specific quiescence for "fa2b1b30b723a7f6097ca46735dfdf5c"
2021-08-11T11:16:15.240+0300 [INFO] (child) spawning: /bin/touch /tmp/test.txt
2021-08-11T11:16:15.240+0300 [DEBUG] (cli) receiving signal "child exited"
2021-08-11T11:16:15.240+0300 [INFO] (runner) child process died
2021-08-11T11:16:15.240+0300 [ERR] (cli) child process died with exit code 0
### Expected behavior
I'm trying to sync web certificates located in Hashicorp Vault to the localhost with Consul Template on it.
To sync multiply KVs to multiply files I'm using savetofile plugin as it would be impossible to create template for each certificate and each domain (to many of them)
When values are changed in Vault they should also get rendered by consul template and get written to the corresponding files
### Actual behavior
With the first run of Consul-Tempalte all values are written to the corresponding files
All further changes in Vault are not rendered.
### Steps to reproduce
1. Launch Consul Template with the provided config and template
2. Check that all KV from the Vault gets rendered to the corresponding folder and file on the localhost (via "savetofile" plugin)
3. Change value inside the Vault
4. Wait for chages to be writen to the cert files
5. Changes are not propagated to the localhost with the Consul Template running
### References
Contributor guide
Research direction
Start with the supplied Vault KV configuration, template, reproduction steps, and debug output, then trace the dynamic dependency and refresh behavior for the secrets listed there. Confirm whether subsequent Vault changes trigger rendering and plugin writes; done means updated KV values reach the corresponding certificate files without restarting Consul Template.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend, devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100