hashicorp / hashicorp/consul-template
Consul Template dry is producing a different result than not-dry
- Dominant language
- Go
- Stars
- 4.8k
- Forks
- 801
- Avg merge
- 4h 5m
- Merged PRs (30d)
- 6
Description
Please note that the Consul Template issue tracker is reserved
for bug reports and enhancements. For general usage questions,
please use the Consul mailing list:
https://groups.google.com/forum/#!forum/consul-tool
### Consul Template version
consul-template v0.19.5 (57b6c71)
### Configuration
(I have removed the commented out lines here for readability)
```hcl
consul {
address = "127.0.0.1:8500"
retry {
enabled = true
attempts = 0
backoff = "250ms"
max_backoff = "30s"
}
}
log_level = "info"
template { source = "/webApps/nginx/conf/env/stage/canary/canary_demo.tmpl" destination = "/webApps/nginx/conf/env/stage/canary/canary_demo.rendered" command = "/bin/sh /webApps/nginx/scripts/reloadNginx.sh" perms = 0777 backup = true wait { min = "2s" max = "10s" } }
template { source = "/webApps/nginx/conf/env/stage/upstreams/consul-upstream.tmpl" destination = "/webApps/nginx/conf/env/stage/upstreams/consul-upstream_conf" perms = 0777 backup = true wait { min = "2s" max = "10s" } }
template { source = "/webApps/nginx/conf/env/stage/locations/consul-locations.tmpl" destination = "/webApps/nginx/conf/env/stage/locations/consul-locations_conf" command = "/bin/sh /webApps/nginx/scripts/reloadNginx.sh" perms = 0777 backup = true wait { min = "2s" max = "10s" } }
```
(I'm only pasting the template that we are having issues with, which is canary_demo.tmpl)
```liquid
# Copy-paste your Consul Template template here
{{ define "canaryHostMap" }}
{{- $appName := . }}
{{- $varSafeAppName := $appName | regexReplaceAll "[^a-zA-Z0-9_]" "_" }}
{{- if keyExists (printf "canary/apps/%s/groupingStrategy" $appName) }}
{{- with $appDetails := tree (printf "canary/apps/%s/" $appName) | explode }}
# appName: {{ $appName }}
# groupingStrategy: {{ $groupingStrategy := index $appDetails "groupingStrategy" }}{{ $groupingStrategy }}
# scaleMode: {{ index $appDetails "scaleMode" }}
map {{ if eq $groupingStrategy "random_user" }}$cookie_aid_3 {{ else }} {{ if eq $groupingStrategy "random_pid" }}$cookie_pid {{ else }}$http_x_correlation_id {{ end }}{{ end }} ${{ $varSafeAppName }}_host {
{{ range service $appName }}
{{- $id := .ID }}
{{- $host := .Address }}
{{- with $versions := index $appDetails "versions" }}
{{- with $version := index $versions $id }}
{{- $groups := index $version "groups" }}
{{- $weight := index $version "weight" }}
{{- $isFeatureCandidate := index $version "feature_candidate" }}
# id: {{ $id }}
# traffic %: {{ $weight }}
# feature_candidate: {{ $isFeatureCandidate }}
{{ range ($groups | split ",") }}{{ . }} {{ $host }};
{{ end }}
{{ end }}
{{ end -}}
{{ else }}# no versions for app{{ end }}
} {{/* end nginx map directive */}}
{{ end }}
{{ else }}
# appName: {{ $appName }}
# no registered services for app
map $http_x_correlation_id ${{ $varSafeAppName }}_host {}
{{ end }}
{{ end }}
{{/* End 'canaryHostMap' definition */}}
{{/*
---------- Everything above this line will be seperated into it's own file ----------
---------- Below is what your canary config will look like ----------
*/}}
{{/* Canary host variable generation */}}
{{ executeTemplate "canaryHostMap" "locator-stage" }}
{{ executeTemplate "canaryHostMap" "item-detail-services-stage" }}
```
### Command
```shell
consul-template -template "/webApps/nginx/conf/env/$PLATFORM_ENVIRONMENT/canary/canary_demo.tmpl:/webApps/nginx/conf/env/$PLATFORM_ENVIRONMENT/canary/canary_demo.rendered" -once
consul-template -template "/webApps/nginx/conf/env/$PLATFORM_ENVIRONMENT/canary/canary_demo.tmpl:/webApps/nginx/conf/env/$PLATFORM_ENVIRONMENT/canary/canary_demo.rendered" -once -dry
```
### Debug output
I think this is the important information
```
2019/04/05 15:10:42.302273 [DEBUG] (runner) initiating run
2019/04/05 15:10:42.302279 [DEBUG] (runner) checking template 456dec88739dd18c1cbfeecd0585ce56
2019/04/05 15:10:42.302695 [DEBUG] (runner) checking template 13a2ab9ad84269b26f8a668350fd197a
2019/04/05 15:10:42.303096 [DEBUG] (runner) checking template 0ab072388fd1c358ed325191ff5c70e4
2019/04/05 15:10:42.303555 [DEBUG] (runner) rendering "/webApps/nginx/conf/env/stage/locations/consul-locations.tmpl" => "/webApps/nginx/conf/env/stage/locations/consul-locations_conf"
2019/04/05 15:10:42.303607 [DEBUG] (runner) diffing and updating dependencies
2019/04/05 15:10:42.303625 [DEBUG] (runner) kv.list(canary/apps/locator-stage/) is still needed
2019/04/05 15:10:42.303632 [DEBUG] (runner) kv.get(canary/apps/item-detail-services-stage/groupingStrategy) is still needed
2019/04/05 15:10:42.303640 [DEBUG] (runner) catalog.services is still needed
2019/04/05 15:10:42.303646 [DEBUG] (runner) kv.get(picking/location) is still needed
2019/04/05 15:10:42.303662 [DEBUG] (runner) health.service(picking|passing) is still needed
2019/04/05 15:10:42.303678 [DEBUG] (runner) kv.get(ecsb/foundation) is still needed
2019/04/05 15:10:42.303690 [DEBUG] (runner) kv.get(picking/stage/external-uri) is still needed
2019/04/05 15:10:42.303703 [DEBUG] (runner) kv.get(canary/apps/locator-stage/groupingStrategy) is still needed
2019/04/05 15:10:42.303720 [DEBUG] (runner) enabling template-specific quiescence for "0ab072388fd1c358ed325191ff5c70e4"
2019/04/05 15:10:42.303740 [DEBUG] (runner) watching 8 dependencies
```
What's weird is that the KV values it says "is still needed" exist in the Consul Server KV. For example:
```
consul kv get canary/apps/locator-stage/groupingStrategy
random_session
```
### Expected behavior
The output from the dry command should match the output to the rendered file
### Actual behavior
Dry output:
```
# appName: locator-stage
# groupingStrategy: random_session
# scaleMode: linear
map $http_x_correlation_id $locator_stage_host {
# id: locator-stage-2-37-3
# traffic %: 1
# feature_candidate: false
default locator-stage-2-37-3.kroger.com;
}
# appName: item-detail-services-stage
# no registered services for app
map $http_x_correlation_id $item_detail_services_stage_host {}
```
File output:
```
# appName: locator-stage
# groupingStrategy: random_session
# scaleMode: linear
map $http_x_correlation_id $locator_stage_host {
# no versions for app
}
# appName: item-detail-services-stage
# no registered services for app
map $http_x_correlation_id $item_detail_services_stage_host {}
```
### Steps to reproduce
1. run `consul-template -template "/webApps/nginx/conf/env/$PLATFORM_ENVIRONMENT/canary/canary_demo.tmpl:/webApps/nginx/conf/env/$PLATFORM_ENVIRONMENT/canary/canary_demo.rendered" -once -dry`
2. run `consul-template -template "/webApps/nginx/conf/env/$PLATFORM_ENVIRONMENT/canary/canary_demo.tmpl:/webApps/nginx/conf/env/$PLATFORM_ENVIRONMENT/canary/canary_demo.rendered" -once`
### References
Are there any other GitHub issues (open or closed) that should
be linked here? For example:
- GH-1234
- ...
N\A
Contributor guide
Research direction
Reproduce the mismatch with the two consul-template commands in the issue, using the supplied HCL configuration and canary_demo.tmpl template. Compare the dry output with the rendered file output, then trace the dry-run rendering path and its handling of the shown Consul KV and service dependencies. Done means both commands produce matching output for this reproduction.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- cli, tooling
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100