hashicorp / hashicorp/consul-template

Support for a semantically poll version which matches consul transactions

Open
#2,026 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
4.8k
Forks
801
Avg merge
4h 5m
Merged PRs (30d)
6

Description

Hi, first of all thanks for keep maintaining this great project!

Ive been playing with `consul-template` and `consul`, and I was trying to push KV changes into my consul templates as quick as possible without having to use the `wait` configuration, while still reducing the chance of having multiple executions of the `command` configured for my template.

For that I was expecting, wrongly - or correct me if Im wrong here - that when making usage of the[ KV transaction API commands](https://developer.hashicorp.com/consul/api-docs/txn) I would be able to push multiple key value changes to consul-template. So for example having an update of two keys, i.e `config/foo` and `config/bar` all together changed in one transaction.

I did some tests with `consul-template` and as it is designed now seems that this can not be guaranteed, since both keys will be watched using independent requests (or am I wrong here?).

I was checking if this behaviour could be guaranteed in somehow, and when using the simple GET command for getting all keys under a specific prefix + the usage of the [index](https://developer.hashicorp.com/consul/api-docs/kv#modifyindex) parameter, i.e

```
curl -X GET "http://localhost:8500/v1/kv/config/?recurse&wait=60s&index=42"
```

When using the TXN Api for updating multiple keys at once, all of them share the same `ModifyIndex`, so Ive tested the previous curl command and I managed to get the expected behaviour, so having one single HTTP response with all keys that I was subscribed with all modifications.

I did not check how the watch is being implemented in Consul, so not sure if this is an intended side effect but if it is this could be used for specifically allow us to batch multiple key updates into one single consul-template reload operation.

am I missing something here?

Contributor guide

Open the contributing guide

Research direction

Start by reviewing how consul-template watches Consul KV prefixes and how Consul's KV and transaction APIs expose ModifyIndex values. Determine whether a semantically polled version can preserve transaction boundaries across multiple keys, and define completion as a documented, tested behavior for batching the resulting template reload.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.