hashicorp / hashicorp/consul-template
initial run returns empty data
- Dominant language
- Go
- Stars
- 4.8k
- Forks
- 801
- Avg merge
- 4h 5m
- Merged PRs (30d)
- 6
Description
### Version
`v0.39.1`
We are using `consul-template` in Exec mode:
```
ExecStart=/usr/local/bin/consul-template \
-config "/var/lib/ssl-proxy-config/template.hcl" \
-template "/var/lib/ssl-proxy-config/proxy.yaml.ctmpl:/var/lib/ssl-proxy-config/proxy.yaml:sudo nginx -s reload"
```
to generate Nginx configuration files dynamically based on service metadata stored in Consul. Our [consul-template](https://gist.github.com/mendelskiv93/3ea053d11c886eb2cf42c5651081d89a) renders tagged services and generates upstream configurations.
Templating is triggered by changes in consul catalog and on each trigger, all ~200 available services are fetched and only ~130 tagged ones are rendered.
For a simplicity, I created a small environment with only 2 tagged services and here is the [trace.log](https://gist.github.com/mendelskiv93/ce794cc5a57689dec7df395d0d35db92).
### Problem
Found a bug while extending our template with plugin which triggers a python script for moving decommissioned sites `.conf `files to a folder.
```go
{{- plugin "/var/lib/ssl-proxy-config/archive.py" (printf "%q" $fqdnList) }}
```
Upon restarting the ssl-proxy service or running consul-template first time even through CLI:
```sh
consul-template -log-level trace -config "/var/lib/ssl-proxy-config/template.hcl" -template "/var/lib/ssl-proxy-config/proxy.yaml.ctmpl:/var/lib/ssl-proxy-config/proxy.yaml:sudo nginx -s reload" -wait=5s:10s
```
the initial fetch cycle renders with empty data.
Looking at the code:
```go
// Fire an initial run to parse all the templates and setup the first-pass
// dependencies. This also forces any templates that have no dependencies to
// be rendered immediately (since they are already renderable).
log.Printf("[DEBUG] (runner) running initial templates")
if err := r.Run(); err != nil {
r.ErrCh <- err
return
}
```
and some other issues:
- https://github.com/hashicorp/consul-template/issues/1637#issuecomment-1372987632
gaves ma an idea of a first-pass run but I cannot really figure out why is this happening when there is a successful response with all services:
```
2024-10-25T08:57:10.797Z [TRACE] catalog.services(@gc-us-central1-a): GET /v1/catalog/services?dc=gc-us-central1-a&stale=true&wait=1m0s
2024-10-25T08:57:10.804Z [TRACE] catalog.services(@do-ams3): returned 190 results
2024-10-25T08:57:10.804Z [TRACE] (view) catalog.services(@do-ams3) marking successful data response
```
before the first rendering:
```
2024-10-25T08:57:11.013Z [DEBUG] (runner) initiating run
2024-10-25T08:57:11.013Z [DEBUG] (runner) checking template 3700978ef4c53efbacef875a5b18923b
2024-10-25T08:57:11.063Z [DEBUG] (runner) rendering "/var/lib/ssl-proxy-config/proxy.yaml.ctmpl" => "/var/lib/ssl-proxy-config/proxy.yaml"
2024-10-25T08:57:11.070Z [INFO] (runner) rendered "/var/lib/ssl-proxy-config/proxy.yaml.ctmpl" => "/var/lib/ssl-proxy-config/proxy.yaml"
2024-10-25T08:57:11.071Z [DEBUG] (runner) appending command ["sudo nginx -s reload"] from "/var/lib/ssl-proxy-config/proxy.yaml.ctmpl" => "/var/lib/ssl-proxy-config/proxy.yaml"
```
### Steps to reproduce
1. Run a consul-template that fetches services from consul catalog
```sh
consul-template -log-level trace -config "/var/lib/ssl-proxy-config/template.hcl" -template "/var/lib/ssl-proxy-config/proxy.yaml.ctmpl:/var/lib/ssl-proxy-config/proxy.yaml:sudo nginx -s reload" -wait=5s:10s
```
2. Use some way of writing every consul-template rendering output to a new file using a [python script](https://gist.github.com/mendelskiv93/fbe5ef5e552fcec9970bfa1784e4e62c) or template itself
My results consistently show the same pattern: an initial empty file followed by a correct file rendered within the same run. After that, `consul-template` continues to run and render normally.
```sh
-rw-r--r-- 1 www-data adm 1 Nov 1 08:24 fqdn_list_20241101_082414.txt
-rw-r--r-- 1 www-data adm 39 Nov 1 08:24 fqdn_list_20241101_082415.txt
.
.
.
-rw-r--r-- 1 www-data adm 39 Nov 1 12:42 fqdn_list_20241101_124244.txt
-rw-r--r-- 1 www-data adm 39 Nov 1 12:43 fqdn_list_20241101_124335.txt
```
Also, using `--wait` seems to have no impact on initial run.
Contributor guide
Assessment
This issue has not been assessed yet.