hashicorp / hashicorp/consul-template

Vault KVs are not dynamically rendered by Consul-Template

Open
#1,504 9 comments 1 reaction 0 assignees View on GitHub
bug unreproducible waiting-reply
Dominant language
Go
Stars
4.8k
Forks
801
Avg merge
4h 5m
Merged PRs (30d)
6

Description

### Consul Template version
v0.26.0 (3b7f233a)
also I'm tried to use https://github.com/ekristen/consul-template-plugin-savetofile plugin

### Configuration

```hcl
# Copy-paste your configuration files here. Only include what is necessary or
# what you've changed from defaults. Include all referenced configurations.
vault {
address = "https://"
token =
unwrap_token = false
renew_token = true
retry {
enabled = true
attempts = 60
backoff = "300ms"
max_backoff = "1m"
}
ssl {
enabled = false
}
}
syslog {
enabled = true
facility = "LOCAL5"
}
deduplicate {
enabled = false
}
```

```liquid
# Copy-paste your Consul Template template here
```
template {
destination = "/local/domain_list.txt"
create_dest_dirs = true
contents = </%s" .) }}{{ range $k, $v := .Data }}
{{ if eq $k "ca.cert" }}
{{ $v | plugin $savetofile "create" (print $path $domain "_ecc/" $k) "1000" "1000" "775" }}
{{ else }}
{{ if eq $k "crt.cert" }}
{{ $v | plugin $savetofile "create" (print $path $domain "_ecc/" $k) "1000" "1000" "775" }}
{{ else }}
{{ if eq $k "priv.key" }}
{{ $v | plugin $savetofile "create" (print $path $domain "_ecc/" $k) "1000" "1000" "775" }}
{{ else }}
{{ if eq $k "fullchain" }}
{{ $v | plugin $savetofile "create" (print $path $domain "_ecc/" $k) "1000" "1000" "755" }}
{{ end }}{{ end }}{{ end }}{{ end }}{{ end }}{{ end }}{{ end }}{{ end }}
EOH
error_on_missing_key = false
perms = 0775
backup = true
wait {
min = "2s"
max = "30s"
}
}
exec {
command = "/bin/systemctl reload nginx"
}

```liquid
# Include sample data you reference in the template from Consul or Vault here.
```

### Debug output
2021-08-11T11:16:15.223+0300 [DEBUG] (cli) receiving signal "child exited"
2021-08-11T11:16:15.224+0300 [DEBUG] (cli) receiving signal "child exited"
2021-08-11T11:16:15.225+0300 [DEBUG] (cli) receiving signal "child exited"
2021-08-11T11:16:15.227+0300 [DEBUG] (cli) receiving signal "child exited"
2021-08-11T11:16:15.228+0300 [DEBUG] (cli) receiving signal "child exited"
2021-08-11T11:16:15.229+0300 [DEBUG] (cli) receiving signal "child exited"
2021-08-11T11:16:15.230+0300 [DEBUG] (cli) receiving signal "child exited"
2021-08-11T11:16:15.231+0300 [DEBUG] (cli) receiving signal "urgent I/O condition"
2021-08-11T11:16:15.232+0300 [DEBUG] (cli) receiving signal "child exited"
2021-08-11T11:16:15.233+0300 [DEBUG] (cli) receiving signal "child exited"
2021-08-11T11:16:15.234+0300 [DEBUG] (cli) receiving signal "child exited"
2021-08-11T11:16:15.235+0300 [DEBUG] (cli) receiving signal "child exited"
2021-08-11T11:16:15.237+0300 [DEBUG] (cli) receiving signal "child exited"
2021-08-11T11:16:15.238+0300 [DEBUG] (cli) receiving signal "child exited"
2021-08-11T11:16:15.239+0300 [DEBUG] (runner) rendering "(dynamic)" => "/local/adtech/certbot/live/domain_list.txt"
2021-08-11T11:16:15.239+0300 [DEBUG] (runner) diffing and updating dependencies
2021-08-11T11:16:15.239+0300 [DEBUG] (runner) vault.read(secret//) is still needed
2021-08-11T11:16:15.239+0300 [DEBUG] (runner) vault.list(secret//) is still needed
2021-08-11T11:16:15.239+0300 [DEBUG] (runner) vault.read(secret//) is still needed
2021-08-11T11:16:15.239+0300 [DEBUG] (runner) vault.read(secret//) is still needed
2021-08-11T11:16:15.239+0300 [DEBUG] (runner) vault.read(secret//) is still needed
2021-08-11T11:16:15.239+0300 [DEBUG] (runner) vault.read(secret//) is still needed
2021-08-11T11:16:15.239+0300 [DEBUG] (runner) vault.read(secret//) is still needed
2021-08-11T11:16:15.239+0300 [DEBUG] (runner) vault.read(secret//) is still needed
.........
2021-08-11T11:16:15.239+0300 [DEBUG] (runner) vault.read(secret//) is still needed
2021-08-11T11:16:15.239+0300 [DEBUG] (runner) vault.read(secret//) is still needed
2021-08-11T11:16:15.239+0300 [DEBUG] (runner) watching 21 dependencies
2021-08-11T11:16:15.239+0300 [DEBUG] (runner) all templates rendered
2021-08-11T11:16:15.240+0300 [DEBUG] (runner) enabling template-specific quiescence for "fa2b1b30b723a7f6097ca46735dfdf5c"
2021-08-11T11:16:15.240+0300 [INFO] (child) spawning: /bin/touch /tmp/test.txt
2021-08-11T11:16:15.240+0300 [DEBUG] (cli) receiving signal "child exited"
2021-08-11T11:16:15.240+0300 [INFO] (runner) child process died
2021-08-11T11:16:15.240+0300 [ERR] (cli) child process died with exit code 0

### Expected behavior
I'm trying to sync web certificates located in Hashicorp Vault to the localhost with Consul Template on it.
To sync multiply KVs to multiply files I'm using savetofile plugin as it would be impossible to create template for each certificate and each domain (to many of them)
When values are changed in Vault they should also get rendered by consul template and get written to the corresponding files

### Actual behavior

With the first run of Consul-Tempalte all values are written to the corresponding files
All further changes in Vault are not rendered.

### Steps to reproduce

1. Launch Consul Template with the provided config and template
2. Check that all KV from the Vault gets rendered to the corresponding folder and file on the localhost (via "savetofile" plugin)
3. Change value inside the Vault
4. Wait for chages to be writen to the cert files
5. Changes are not propagated to the localhost with the Consul Template running

### References

Contributor guide

Open the contributing guide

Research direction

Start with the supplied Vault KV configuration, template, reproduction steps, and debug output, then trace the dynamic dependency and refresh behavior for the secrets listed there. Confirm whether subsequent Vault changes trigger rendering and plugin writes; done means updated KV values reach the corresponding certificate files without restarting Consul Template.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend, devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.