hashicorp / hashicorp/consul-template

Allowing rendering a template even when reading an optional secret fails

Open
#1,836 0 comments 8 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
4.8k
Forks
801
Avg merge
4h 5m
Merged PRs (30d)
6

Description

When reading multiple secrets from Vault in the same template, if reading one of those secrets fails then the template's execution fails and the templated file is never created/updated. It would be nice to have a function (eg: `optionalSecret`) that allows the template's execution to continue even if reading one of the secrets fails.

For instance, let's say we have three dynamic secrets in Vault. Each secret provides credentials for a different database, and each database runs in a separate failure domain. We read those secrets with the Vault Agent, which uses the Consul template library:

```liquid
{{- with secret "database_alpha/creds/read-only" }}
alpha:
username: {{ .Data.username }}
password: {{ .Data.password }}
{{- end }}
{{- with secret "database_bravo/creds/read-only" }}
bravo:
username: {{ .Data.username }}
password: {{ .Data.password }}
{{- end }}
{{- with secret "database_charlie/creds/read-only" }}
charlie:
username: {{ .Data.username }}
password: {{ .Data.password }}
{{- end }}
```

Executing the template calls the `secret` function three times, each sending a request to Vault. If all databases are available then the template executes successfully and the Vault Agent creates/updates the template file. However if one of the databases is unavailable then Vault will respond to the corresponding request with a 500 error. The call to `secret` will return an error and the template's execution will fail. The Vault Agent will not create/update the templated file.

In our use case, we want to tolerate a single database failing, so that our services can still connect to the other two databases. Would you consider adding a new function to the Consul template library that allows for finer error handling?

For example, we could have a `optionalSecret` function that acts like the `secret` function except that it does not return an error if reading the secret fails. Instead, the function could return a nil value. The template from above would look something like this:

```liquid
{{- with optionalSecret "database_alpha/creds/read-only" }}
alpha:
username: {{ .Data.username }}
password: {{ .Data.password }}
{{- end }}
{{- with optionalSecret "database_bravo/creds/read-only" }}
bravo:
username: {{ .Data.username }}
password: {{ .Data.password }}
{{- end }}
{{- with optionalSecret "database_charlie/creds/read-only" }}
charlie:
username: {{ .Data.username }}
password: {{ .Data.password }}
{{- end }}
```

This would also open the door to finer error handling:

```liquid
{{- with optionalSecret "database_alpha/creds/read-only" }}
alpha:
username: {{ .Data.username }}
password: {{ .Data.password }}
{{- else }}
alpha:
error: failed to get credentials, check app dashboard for details
{{- end }}
```

In principle, this is related to the `secretOrDefault` function described in issue #942. That issue describes a use case with non-existent secrets, which is a little different from our use case where the secrets exist but reading them fails.

One could work around this by writing each secret to a separate file: an unavailable database would only prevent a single file from being created/updated. However that doesn't match our use case: the list of secrets to fetch is obtained dynamically with the `secrets` function, so we need to use a single template.

What do you think about adding the `optionalSecret` function described above, or something similar?

Contributor guide

Open the contributing guide

Research direction

Start by inspecting the existing `secret` and `secrets` template functions and how their errors affect template execution. Determine the intended behavior for an `optionalSecret`-style function, including failed reads and `with`/`else` handling, then add tests showing that one failed secret does not prevent the remaining template from rendering.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.