hashicorp / hashicorp/consul-replicate

Recommendation for gathering "replication lag" status?

Open
#81 0 comments 2 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
524
Forks
72
PR merge metrics
No merged PRs in 30d

Description

Hello, I wanted to see if there is a recommended method to show an indication of replication lag or "freshness" for Consul data that is copied with consul-replicate? The intent is to show that the "source" shows a `ModifyIndex` of e.g. 1000 while the "destination" contains only e.g. 998.

We are experimenting with the following approach, which is quite awkward.

1. On the "destination", read from Consul the encoded status recorded by consul-replicate, and get the `LastReplicated` value
2. On the "source", dump all KV data and find the maximum `ModifyIndex`
3. Compare the LastReplicated and ModifyIndex values

Pseudo-commands for #1 and #2:

```
destination $ consul kv export service/consul-replicate/statuses | jq -r '.[0].value' | base64 -d | jq '.LastReplicated'
source $ curl -s localhost:8500/v1/kv/global?recurse=true | jq -r ' . | max_by(.ModifyIndex) | .ModifyIndex'"
```

Ideally, `consul-replicate` would track an equivalent of the above itself, and report the replication lag within a status message, preferably in a discrete value (non-base64) to make for easy monitoring & alerting.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.