influxdata / influxdata/telegraf
MQTT input sent to two different outputs completely stops reporting if one output unresponsive
- Dominant language
- Go
- Stars
- 17.8k
- Forks
- 5.8k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 161
Description
### Relevant telegraf.conf
```toml
[agent]
debug = true
quiet = false
interval = "10s"
### --- setup for just mocks ---- ###
[[inputs.mock]]
metric_name = "fake_alpha"
[inputs.mock.tags]
destination_topic = "alpha"
[[inputs.mock.constant]]
name = "fake_val"
value = "1234"
[[inputs.mock]]
metric_name = "fake_beta"
[inputs.mock.tags]
destination_topic = "beta"
[[inputs.mock.constant]]
name = "fake_val"
value = "5678"
[[outputs.mqtt]]
client_id = "fake_data_client"
namepass = ["fake*"]
servers = ["mqtt://mosquitto:1883"]
data_format = "json"
topic = "{{ .Tag \"destination_topic\" }}"
### -- end setup for mock data -- ###
[[inputs.mqtt_consumer]]
servers = ["mqtt://mosquitto:1883"]
data_format = "json"
topics = ["alpha"]
name_override = "alpha_mqtt"
[[inputs.mqtt_consumer]]
servers = ["mqtt://mosquitto:1883"]
data_format = "json"
topics = ["beta"]
name_override = "beta_mqtt"
max_undelivered_messages = 10
[[outputs.file]]
files = ["stdout"]
data_format = "json"
namepass = ["alpha_mqtt", "beta_mqtt"]
[[outputs.http]]
url = "http://notarealsite1234.com"
method = "PUT"
data_format = "json"
metric_batch_size = 1
```
### Logs from Telegraf
```text
2025-02-20T18:27:25Z I! Loading config: /etc/telegraf/telegraf.conf
2025-02-20T18:27:25Z I! Starting Telegraf 1.33.2 brought to you by InfluxData the makers of InfluxDB
2025-02-20T18:27:25Z I! Available plugins: 236 inputs, 9 aggregators, 33 processors, 26 parsers, 63 outputs, 6 secret-stores
2025-02-20T18:27:25Z I! Loaded inputs: mock (2x) mqtt_consumer (2x)
2025-02-20T18:27:25Z I! Loaded aggregators:
2025-02-20T18:27:25Z I! Loaded processors:
2025-02-20T18:27:25Z I! Loaded secretstores:
2025-02-20T18:27:25Z I! Loaded outputs: file http mqtt
2025-02-20T18:27:25Z I! Tags enabled: host=654f1341740f
2025-02-20T18:27:25Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:"654f1341740f", Flush Interval:10s
2025-02-20T18:27:25Z W! [agent] The default value of 'skip_processors_after_aggregators' will change to 'true' with Telegraf v1.40.0! If you need the current default behavior, please explicitly set the option to 'false'!
2025-02-20T18:27:25Z D! [agent] Initializing plugins
2025-02-20T18:27:25Z D! [agent] Connecting outputs
2025-02-20T18:27:25Z D! [agent] Attempting connection to [outputs.http]
2025-02-20T18:27:25Z D! [agent] Successfully connected to outputs.http
2025-02-20T18:27:25Z D! [agent] Attempting connection to [outputs.mqtt]
2025-02-20T18:27:25Z D! [agent] Successfully connected to outputs.mqtt
2025-02-20T18:27:25Z D! [agent] Attempting connection to [outputs.file]
2025-02-20T18:27:25Z D! [agent] Successfully connected to outputs.file
2025-02-20T18:27:25Z D! [agent] Starting service inputs
2025-02-20T18:27:25Z I! [inputs.mqtt_consumer] Connected [mqtt://mosquitto:1883]
2025-02-20T18:27:25Z I! [inputs.mqtt_consumer] Connected [mqtt://mosquitto:1883]
2025-02-20T18:27:30Z D! [outputs.http] Buffer fullness: 2 / 10000 metrics
2025-02-20T18:27:30Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:27:30Z D! [outputs.http] Buffer fullness: 2 / 10000 metrics
2025-02-20T18:27:30Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:27:35Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:27:35Z D! [outputs.mqtt] Wrote batch of 2 metrics in 685.2µs
2025-02-20T18:27:35Z D! [outputs.mqtt] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:27:35Z D! [outputs.http] Buffer fullness: 4 / 10000 metrics
2025-02-20T18:27:35Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:27:35Z D! [outputs.http] Buffer fullness: 4 / 10000 metrics
2025-02-20T18:27:35Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:27:39Z D! [outputs.http] Buffer fullness: 6 / 10000 metrics
2025-02-20T18:27:39Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:27:39Z D! [outputs.http] Buffer fullness: 6 / 10000 metrics
2025-02-20T18:27:39Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:27:45Z D! [outputs.file] Wrote batch of 2 metrics in 153.9µs
2025-02-20T18:27:45Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:27:45Z D! [outputs.mqtt] Wrote batch of 2 metrics in 563.5µs
2025-02-20T18:27:45Z D! [outputs.mqtt] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:27:45Z D! [outputs.http] Buffer fullness: 8 / 10000 metrics
2025-02-20T18:27:45Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:27:45Z D! [outputs.http] Buffer fullness: 8 / 10000 metrics
2025-02-20T18:27:45Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:27:50Z D! [outputs.http] Buffer fullness: 10 / 10000 metrics
2025-02-20T18:27:50Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:27:50Z D! [outputs.http] Buffer fullness: 10 / 10000 metrics
2025-02-20T18:27:50Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:27:55Z D! [outputs.file] Wrote batch of 2 metrics in 144.2µs
2025-02-20T18:27:55Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:27:55Z D! [outputs.mqtt] Wrote batch of 2 metrics in 592.3µs
2025-02-20T18:27:55Z D! [outputs.mqtt] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:27:55Z D! [outputs.http] Buffer fullness: 12 / 10000 metrics
2025-02-20T18:27:55Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:27:55Z D! [outputs.http] Buffer fullness: 12 / 10000 metrics
2025-02-20T18:27:55Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:28:00Z D! [outputs.http] Buffer fullness: 14 / 10000 metrics
2025-02-20T18:28:00Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:28:00Z D! [outputs.http] Buffer fullness: 14 / 10000 metrics
2025-02-20T18:28:00Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:28:05Z D! [outputs.file] Wrote batch of 2 metrics in 192.6µs
2025-02-20T18:28:05Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:28:05Z D! [outputs.mqtt] Wrote batch of 2 metrics in 511.4µs
2025-02-20T18:28:05Z D! [outputs.mqtt] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:28:05Z D! [outputs.http] Buffer fullness: 16 / 10000 metrics
2025-02-20T18:28:05Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:28:05Z D! [outputs.http] Buffer fullness: 16 / 10000 metrics
2025-02-20T18:28:05Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:28:09Z D! [outputs.http] Buffer fullness: 18 / 10000 metrics
2025-02-20T18:28:09Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:28:09Z D! [outputs.http] Buffer fullness: 18 / 10000 metrics
2025-02-20T18:28:09Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:28:15Z D! [outputs.file] Wrote batch of 2 metrics in 192µs
2025-02-20T18:28:15Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:28:15Z D! [outputs.mqtt] Wrote batch of 2 metrics in 970.8µs
2025-02-20T18:28:15Z D! [outputs.mqtt] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:28:15Z D! [outputs.http] Buffer fullness: 20 / 10000 metrics
2025-02-20T18:28:15Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:28:15Z D! [outputs.http] Buffer fullness: 20 / 10000 metrics
2025-02-20T18:28:15Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:28:20Z D! [outputs.http] Buffer fullness: 22 / 10000 metrics
2025-02-20T18:28:20Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:28:20Z D! [outputs.http] Buffer fullness: 22 / 10000 metrics
2025-02-20T18:28:20Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:28:25Z D! [outputs.file] Wrote batch of 2 metrics in 115.7µs
2025-02-20T18:28:25Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:28:25Z D! [outputs.mqtt] Wrote batch of 2 metrics in 406.9µs
2025-02-20T18:28:25Z D! [outputs.mqtt] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:28:25Z D! [outputs.http] Buffer fullness: 24 / 10000 metrics
2025-02-20T18:28:25Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:28:25Z D! [outputs.http] Buffer fullness: 24 / 10000 metrics
2025-02-20T18:28:25Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:28:30Z D! [outputs.http] Buffer fullness: 26 / 10000 metrics
2025-02-20T18:28:30Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:28:30Z D! [outputs.http] Buffer fullness: 26 / 10000 metrics
2025-02-20T18:28:30Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:28:35Z D! [outputs.file] Wrote batch of 2 metrics in 125µs
2025-02-20T18:28:35Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:28:35Z D! [outputs.mqtt] Wrote batch of 2 metrics in 551µs
2025-02-20T18:28:35Z D! [outputs.mqtt] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:28:35Z D! [outputs.http] Buffer fullness: 28 / 10000 metrics
2025-02-20T18:28:35Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:28:35Z D! [outputs.http] Buffer fullness: 28 / 10000 metrics
2025-02-20T18:28:35Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:28:39Z D! [outputs.http] Buffer fullness: 30 / 10000 metrics
2025-02-20T18:28:39Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:28:39Z D! [outputs.http] Buffer fullness: 30 / 10000 metrics
2025-02-20T18:28:39Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:28:45Z D! [outputs.file] Wrote batch of 2 metrics in 220.9µs
2025-02-20T18:28:45Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:28:45Z D! [outputs.mqtt] Wrote batch of 2 metrics in 643.6µs
2025-02-20T18:28:45Z D! [outputs.mqtt] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:28:45Z D! [outputs.http] Buffer fullness: 32 / 10000 metrics
2025-02-20T18:28:45Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:28:45Z D! [outputs.http] Buffer fullness: 32 / 10000 metrics
2025-02-20T18:28:45Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:28:50Z D! [outputs.http] Buffer fullness: 34 / 10000 metrics
2025-02-20T18:28:50Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:28:50Z D! [outputs.http] Buffer fullness: 34 / 10000 metrics
2025-02-20T18:28:50Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:28:55Z D! [outputs.file] Wrote batch of 2 metrics in 199.9µs
2025-02-20T18:28:55Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:28:55Z D! [outputs.mqtt] Wrote batch of 2 metrics in 977.5µs
2025-02-20T18:28:55Z D! [outputs.mqtt] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:28:55Z D! [outputs.http] Buffer fullness: 36 / 10000 metrics
2025-02-20T18:28:55Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:28:55Z D! [outputs.http] Buffer fullness: 36 / 10000 metrics
{"fields":{"timestamp":1740076050},"name":"beta_mqtt","tags":{"host":"654f1341740f","topic":"beta"},"timestamp":1740076055}
{"fields":{"timestamp":1740076050},"name":"alpha_mqtt","tags":{"host":"654f1341740f","topic":"alpha"},"timestamp":1740076055}
{"fields":{"timestamp":1740076060},"name":"alpha_mqtt","tags":{"host":"654f1341740f","topic":"alpha"},"timestamp":1740076065}
{"fields":{"timestamp":1740076060},"name":"beta_mqtt","tags":{"host":"654f1341740f","topic":"beta"},"timestamp":1740076065}
{"fields":{"timestamp":1740076070},"name":"alpha_mqtt","tags":{"host":"654f1341740f","topic":"alpha"},"timestamp":1740076075}
{"fields":{"timestamp":1740076070},"name":"beta_mqtt","tags":{"host":"654f1341740f","topic":"beta"},"timestamp":1740076075}
{"fields":{"timestamp":1740076080},"name":"beta_mqtt","tags":{"host":"654f1341740f","topic":"beta"},"timestamp":1740076085}
{"fields":{"timestamp":1740076080},"name":"alpha_mqtt","tags":{"host":"654f1341740f","topic":"alpha"},"timestamp":1740076085}
{"fields":{"timestamp":1740076090},"name":"alpha_mqtt","tags":{"host":"654f1341740f","topic":"alpha"},"timestamp":1740076095}
{"fields":{"timestamp":1740076090},"name":"beta_mqtt","tags":{"host":"654f1341740f","topic":"beta"},"timestamp":1740076095}
{"fields":{"timestamp":1740076100},"name":"beta_mqtt","tags":{"host":"654f1341740f","topic":"beta"},"timestamp":1740076105}
{"fields":{"timestamp":1740076100},"name":"alpha_mqtt","tags":{"host":"654f1341740f","topic":"alpha"},"timestamp":1740076105}
{"fields":{"timestamp":1740076110},"name":"beta_mqtt","tags":{"host":"654f1341740f","topic":"beta"},"timestamp":1740076115}
{"fields":{"timestamp":1740076110},"name":"alpha_mqtt","tags":{"host":"654f1341740f","topic":"alpha"},"timestamp":1740076115}
{"fields":{"timestamp":1740076120},"name":"alpha_mqtt","tags":{"host":"654f1341740f","topic":"alpha"},"timestamp":1740076125}
{"fields":{"timestamp":1740076120},"name":"beta_mqtt","tags":{"host":"654f1341740f","topic":"beta"},"timestamp":1740076125}
{"fields":{"timestamp":1740076130},"name":"beta_mqtt","tags":{"host":"654f1341740f","topic":"beta"},"timestamp":1740076135}
{"fields":{"timestamp":1740076130},"name":"alpha_mqtt","tags":{"host":"654f1341740f","topic":"alpha"},"timestamp":1740076135}
{"fields":{"timestamp":1740076140},"name":"beta_mqtt","tags":{"host":"654f1341740f","topic":"beta"},"timestamp":1740076145}
{"fields":{"timestamp":1740076140},"name":"alpha_mqtt","tags":{"host":"654f1341740f","topic":"alpha"},"timestamp":1740076145}
{"fields":{"timestamp":1740076150},"name":"alpha_mqtt","tags":{"host":"654f1341740f","topic":"alpha"},"timestamp":1740076155}
{"fields":{"timestamp":1740076160},"name":"alpha_mqtt","tags":{"host":"654f1341740f","topic":"alpha"},"timestamp":1740076165}
{"fields":{"timestamp":1740076170},"name":"alpha_mqtt","tags":{"host":"654f1341740f","topic":"alpha"},"timestamp":1740076175}
2025-02-20T18:28:55Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:29:00Z D! [outputs.http] Buffer fullness: 38 / 10000 metrics
2025-02-20T18:29:00Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:29:00Z D! [outputs.http] Buffer fullness: 38 / 10000 metrics
2025-02-20T18:29:00Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:29:05Z D! [outputs.file] Wrote batch of 2 metrics in 153.7µs
2025-02-20T18:29:05Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:29:05Z D! [outputs.mqtt] Wrote batch of 2 metrics in 655.3µs
2025-02-20T18:29:05Z D! [outputs.mqtt] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:29:05Z D! [outputs.http] Buffer fullness: 40 / 10000 metrics
2025-02-20T18:29:05Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:29:05Z D! [outputs.http] Buffer fullness: 40 / 10000 metrics
2025-02-20T18:29:05Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:29:09Z D! [outputs.http] Buffer fullness: 42 / 10000 metrics
2025-02-20T18:29:09Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:29:09Z D! [outputs.http] Buffer fullness: 42 / 10000 metrics
2025-02-20T18:29:09Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:29:15Z D! [outputs.file] Wrote batch of 2 metrics in 212.7µs
2025-02-20T18:29:15Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:29:15Z D! [outputs.mqtt] Wrote batch of 2 metrics in 812.3µs
2025-02-20T18:29:15Z D! [outputs.mqtt] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:29:15Z D! [outputs.http] Buffer fullness: 43 / 10000 metrics
2025-02-20T18:29:15Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:29:15Z D! [outputs.http] Buffer fullness: 43 / 10000 metrics
2025-02-20T18:29:15Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:29:20Z D! [outputs.http] Buffer fullness: 45 / 10000 metrics
2025-02-20T18:29:20Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:29:20Z D! [outputs.http] Buffer fullness: 45 / 10000 metrics
2025-02-20T18:29:20Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:29:25Z D! [outputs.file] Wrote batch of 1 metrics in 100µs
2025-02-20T18:29:25Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:29:25Z D! [outputs.mqtt] Wrote batch of 2 metrics in 518.5µs
2025-02-20T18:29:25Z D! [outputs.mqtt] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:29:25Z D! [outputs.http] Buffer fullness: 46 / 10000 metrics
2025-02-20T18:29:25Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:29:25Z D! [outputs.http] Buffer fullness: 46 / 10000 metrics
2025-02-20T18:29:25Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:29:30Z D! [outputs.http] Buffer fullness: 48 / 10000 metrics
2025-02-20T18:29:30Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:29:30Z D! [outputs.http] Buffer fullness: 48 / 10000 metrics
2025-02-20T18:29:30Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:29:35Z D! [outputs.file] Wrote batch of 1 metrics in 109µs
2025-02-20T18:29:35Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:29:35Z D! [outputs.mqtt] Wrote batch of 2 metrics in 565.1µs
2025-02-20T18:29:35Z D! [outputs.mqtt] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:29:35Z D! [outputs.http] Buffer fullness: 49 / 10000 metrics
2025-02-20T18:29:35Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:29:35Z D! [outputs.http] Buffer fullness: 49 / 10000 metrics
2025-02-20T18:29:35Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:29:39Z D! [outputs.http] Buffer fullness: 51 / 10000 metrics
2025-02-20T18:29:39Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:29:39Z D! [outputs.http] Buffer fullness: 51 / 10000 metrics
2025-02-20T18:29:39Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:29:45Z D! [outputs.file] Wrote batch of 1 metrics in 99.8µs
2025-02-20T18:29:45Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:29:45Z D! [outputs.mqtt] Wrote batch of 2 metrics in 453.7µs
2025-02-20T18:29:45Z D! [outputs.mqtt] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:29:45Z D! [outputs.http] Buffer fullness: 52 / 10000 metrics
2025-02-20T18:29:45Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:29:45Z D! [outputs.http] Buffer fullness: 52 / 10000 metrics
2025-02-20T18:29:45Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:29:50Z D! [outputs.http] Buffer fullness: 54 / 10000 metrics
2025-02-20T18:29:50Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:29:50Z D! [outputs.http] Buffer fullness: 54 / 10000 metrics
2025-02-20T18:29:50Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:29:55Z D! [outputs.file] Wrote batch of 1 metrics in 124.8µs
2025-02-20T18:29:55Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:29:55Z D! [outputs.mqtt] Wrote batch of 2 metrics in 885.5µs
2025-02-20T18:29:55Z D! [outputs.mqtt] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:29:55Z D! [outputs.http] Buffer fullness: 55 / 10000 metrics
2025-02-20T18:29:55Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:29:55Z D! [outputs.http] Buffer fullness: 55 / 10000 metrics
2025-02-20T18:29:55Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:30:00Z D! [outputs.http] Buffer fullness: 57 / 10000 metrics
2025-02-20T18:30:00Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:30:00Z D! [outputs.http] Buffer fullness: 57 / 10000 metrics
{"fields":{"timestamp":1740076180},"name":"alpha_mqtt","tags":{"host":"654f1341740f","topic":"alpha"},"timestamp":1740076185}
{"fields":{"timestamp":1740076190},"name":"alpha_mqtt","tags":{"host":"654f1341740f","topic":"alpha"},"timestamp":1740076195}
{"fields":{"timestamp":1740076200},"name":"alpha_mqtt","tags":{"host":"654f1341740f","topic":"alpha"},"timestamp":1740076205}
{"fields":{"timestamp":1740076210},"name":"alpha_mqtt","tags":{"host":"654f1341740f","topic":"alpha"},"timestamp":1740076215}
2025-02-20T18:30:00Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:30:05Z D! [outputs.file] Wrote batch of 1 metrics in 123.3µs
2025-02-20T18:30:05Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:30:05Z D! [outputs.mqtt] Wrote batch of 2 metrics in 716.5µs
2025-02-20T18:30:05Z D! [outputs.mqtt] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:30:05Z D! [outputs.http] Buffer fullness: 58 / 10000 metrics
2025-02-20T18:30:05Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:30:05Z D! [outputs.http] Buffer fullness: 58 / 10000 metrics
2025-02-20T18:30:05Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:30:09Z D! [outputs.http] Buffer fullness: 60 / 10000 metrics
2025-02-20T18:30:09Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:30:09Z D! [outputs.http] Buffer fullness: 60 / 10000 metrics
2025-02-20T18:30:09Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:30:15Z D! [outputs.file] Wrote batch of 1 metrics in 90.8µs
2025-02-20T18:30:15Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:30:15Z D! [outputs.mqtt] Wrote batch of 2 metrics in 644.3µs
2025-02-20T18:30:15Z D! [outputs.mqtt] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:30:15Z D! [outputs.http] Buffer fullness: 61 / 10000 metrics
2025-02-20T18:30:15Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:30:15Z D! [outputs.http] Buffer fullness: 61 / 10000 metrics
2025-02-20T18:30:15Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:30:20Z D! [outputs.http] Buffer fullness: 63 / 10000 metrics
2025-02-20T18:30:20Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:30:20Z D! [outputs.http] Buffer fullness: 63 / 10000 metrics
2025-02-20T18:30:20Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:30:25Z D! [outputs.file] Wrote batch of 1 metrics in 117.3µs
2025-02-20T18:30:25Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:30:25Z D! [outputs.mqtt] Wrote batch of 2 metrics in 905.4µs
2025-02-20T18:30:25Z D! [outputs.mqtt] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:30:25Z D! [outputs.http] Buffer fullness: 64 / 10000 metrics
2025-02-20T18:30:25Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:30:25Z D! [outputs.http] Buffer fullness: 64 / 10000 metrics
2025-02-20T18:30:25Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:30:30Z D! [outputs.http] Buffer fullness: 66 / 10000 metrics
2025-02-20T18:30:30Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:30:30Z D! [outputs.http] Buffer fullness: 66 / 10000 metrics
2025-02-20T18:30:30Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:30:35Z D! [outputs.file] Wrote batch of 1 metrics in 81.8µs
2025-02-20T18:30:35Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:30:35Z D! [outputs.mqtt] Wrote batch of 2 metrics in 594.8µs
2025-02-20T18:30:35Z D! [outputs.mqtt] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:30:35Z D! [outputs.http] Buffer fullness: 67 / 10000 metrics
2025-02-20T18:30:35Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:30:35Z D! [outputs.http] Buffer fullness: 67 / 10000 metrics
2025-02-20T18:30:35Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:30:39Z D! [outputs.http] Buffer fullness: 69 / 10000 metrics
2025-02-20T18:30:39Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:30:39Z D! [outputs.http] Buffer fullness: 69 / 10000 metrics
2025-02-20T18:30:39Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:30:45Z D! [outputs.file] Wrote batch of 1 metrics in 102.3µs
2025-02-20T18:30:45Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:30:45Z D! [outputs.mqtt] Wrote batch of 2 metrics in 876.4µs
2025-02-20T18:30:45Z D! [outputs.mqtt] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:30:45Z D! [outputs.http] Buffer fullness: 70 / 10000 metrics
2025-02-20T18:30:45Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:30:45Z D! [outputs.http] Buffer fullness: 70 / 10000 metrics
2025-02-20T18:30:45Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:30:50Z D! [inputs.mqtt_consumer] Connecting [mqtt://mosquitto:1883]
2025-02-20T18:30:50Z I! [inputs.mqtt_consumer] Connected [mqtt://mosquitto:1883]
2025-02-20T18:30:50Z D! [outputs.http] Buffer fullness: 72 / 10000 metrics
2025-02-20T18:30:50Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:30:50Z D! [outputs.http] Buffer fullness: 72 / 10000 metrics
2025-02-20T18:30:50Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:30:55Z D! [outputs.file] Wrote batch of 1 metrics in 233.5µs
2025-02-20T18:30:55Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:30:55Z D! [outputs.mqtt] Wrote batch of 2 metrics in 756.1µs
2025-02-20T18:30:55Z D! [outputs.mqtt] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:30:55Z D! [outputs.http] Buffer fullness: 73 / 10000 metrics
2025-02-20T18:30:55Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:30:55Z D! [outputs.http] Buffer fullness: 73 / 10000 metrics
2025-02-20T18:30:55Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:31:00Z D! [outputs.http] Buffer fullness: 75 / 10000 metrics
{"fields":{"timestamp":1740076220},"name":"alpha_mqtt","tags":{"host":"654f1341740f","topic":"alpha"},"timestamp":1740076225}
{"fields":{"timestamp":1740076230},"name":"alpha_mqtt","tags":{"host":"654f1341740f","topic":"alpha"},"timestamp":1740076235}
{"fields":{"timestamp":1740076240},"name":"alpha_mqtt","tags":{"host":"654f1341740f","topic":"alpha"},"timestamp":1740076245}
{"fields":{"timestamp":1740076250},"name":"alpha_mqtt","tags":{"host":"654f1341740f","topic":"alpha"},"timestamp":1740076255}
{"fields":{"timestamp":1740076260},"name":"alpha_mqtt","tags":{"host":"654f1341740f","topic":"alpha"},"timestamp":1740076265}
2025-02-20T18:31:00Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:31:00Z D! [outputs.http] Buffer fullness: 75 / 10000 metrics
2025-02-20T18:31:00Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:31:05Z D! [outputs.file] Wrote batch of 1 metrics in 127.9µs
2025-02-20T18:31:05Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:31:05Z D! [outputs.mqtt] Wrote batch of 2 metrics in 692µs
2025-02-20T18:31:05Z D! [outputs.mqtt] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:31:05Z D! [outputs.http] Buffer fullness: 76 / 10000 metrics
2025-02-20T18:31:05Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:31:05Z D! [outputs.http] Buffer fullness: 76 / 10000 metrics
2025-02-20T18:31:05Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:31:05Z D! [agent] Stopping service inputs
2025-02-20T18:31:05Z D! [inputs.mqtt_consumer] Disconnecting [mqtt://mosquitto:1883]
2025-02-20T18:31:05Z D! [inputs.mqtt_consumer] Disconnected [mqtt://mosquitto:1883]
2025-02-20T18:31:05Z D! [inputs.mqtt_consumer] Disconnecting [mqtt://mosquitto:1883]
2025-02-20T18:31:06Z D! [inputs.mqtt_consumer] Disconnected [mqtt://mosquitto:1883]
2025-02-20T18:31:06Z D! [agent] Input channel closed
2025-02-20T18:31:06Z I! [agent] Hang on, flushing any cached metrics before shutdown
2025-02-20T18:31:06Z D! [outputs.file] Wrote batch of 1 metrics in 76.3µs
2025-02-20T18:31:06Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:31:06Z D! [outputs.mqtt] Buffer fullness: 0 / 10000 metrics
2025-02-20T18:31:06Z D! [outputs.http] Buffer fullness: 76 / 10000 metrics
2025-02-20T18:31:06Z E! [agent] Error writing to outputs.http: Put "http://notarealsite1234.com": dial tcp: lookup notarealsite1234.com on 127.0.0.11:53: no such host
2025-02-20T18:31:06Z I! [agent] Stopping running outputs
2025-02-20T18:31:06Z D! [agent] Stopped Successfully
```
### System info
Official Telegraf Docker container for 1.32 and 1.33, Mosquitto docker container 2.0, Docker 20.10.12
### Docker
```
version: '3.6'
services:
telegraf:
container_name: telegraf
image: telegraf:1.33
volumes:
- ./telegraf.conf:/etc/telegraf/telegraf.conf
restart: always
depends_on:
- mosquitto
command: ["telegraf", "--config", "/etc/telegraf/telegraf.conf", "--debug"]
mosquitto:
container_name: mosquitto
image: eclipse-mosquitto:2.0
volumes:
- ./mosquitto.conf:/mosquitto/config/mosquitto.conf
ports:
- 1883:1883
```
### Steps to reproduce
1. Have two MQTT consumers (A & B) listening on different topics. MQTT consumer B has a `max_undelivered_messages` set to a low number (in my example, it's set to 10)
2. Have both MQTT consumers sending to TWO outputs, **one of which is unresponsive / will fail to write**, and one which we'll call "good"
3. Send constant info over MQTT to both MQTT topics for A & B
4. Monitor "good" output to make sure data from A & B are flowing
5. Wait for MQTT B consumer to disconnect + reconnect
6. Monitor "good" output for data from A & B
### Expected behavior
Data from A & B continues to send to both outputs. After B disconnects, I expect it to reconnect and resume sending MQTT B messages to the "good" output.
### Actual behavior
After B disconnects, it seems to reconnect but does not send any more data to outputs.
### Additional info
This is not a resource issue, as I've confirmed that my docker containers are not even close to running out of memory / CPU. Attached is a diagram showing how I set up my telegraf.conf to reproduce the issue, but we were seeing this issue with independent MQTT messages coming in production.

Here is a link to my Github repo with a docker compose, mosquitto.conf, and telegraf.conf that repro the issue reliably. https://github.com/annamooseity/telegraf-mqtt-weirdness
Contributor guide
Research direction
Reproduce the supplied configuration with the mqtt_consumer inputs and http, mqtt, and file outputs, using the logs to observe buffering and delivery. Trace the agent path handling a failing outputs.http write alongside successful outputs.file and outputs.mqtt writes; done means one unresponsive output no longer prevents the MQTT inputs from continuing to report.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100