hashicorp / hashicorp/envconsul

Too often requests at 404 error, do not respecting retry settings

Open
#153 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Go
Stars
2.1k
Forks
194
PR merge metrics
No merged PRs in 30d

Description

### Envconsul version
envconsul v0.7.2 (a508c43)

### Command
```shell
./envconsul -consul-addr=http://10.10.12.84:8500 -prefix any_word -log-level trace watch env
```

### Expected behavior
At any error, including http 4XX, use retry setting (Attempts+Backoff+MaxBackoff) if retry enabled

### Actual behavior
http requests storm at 404 error if envconsul pointed to any web server

### Steps to reproduce
- always: if **-consul-addr** point to reverse proxy(nginx) and consul upstream down
- sometime: if **-consul-addr** point to consul agent (usually at consul agent stop/start time)

1. run in terminal 1
```shell
docker run --name nginx --rm -it -p 80:80 nginx | wc -l
```
2. run in terminal 2
```shell
timeout -k 1 30 ./envconsul -consul-addr=http://localhost -prefix any_word watch env
```
3. in terminal 2 wait 30 seconds for "Killed"
4. run in terminal 2
```shell
docker stop nginx
```
5. watch to terminal 1
```shell
79798
```
80k/30s ~= 2600 requests per second 👎 , 80k requests total,

When point to real consul agent - only one request during 30s
```shell
docker run --rm -it -p 8500:8500 consul agent -dev -ui -client=0.0.0.0 | grep http | wc -l
timeout -k 1 30 ./envconsul -consul-addr=http://localhost:8500 -prefix any_word watch env
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.