influxdata / influxdata/telegraf
FR: inputs.http could support streaming with chunked encoding
- Dominant language
- Go
- Stars
- 17.8k
- Forks
- 5.8k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 161
Description
### Use Case
I currently try to stream metrics in influx line protocol from one telegraf to the next. The receiving telegraf opens the connection to a python-script-webserver (outputs.execd) with bearer token authentication.
This all works well and good, but I have to batch the data and the requesting telegraf is tied to an interval in which there might be more or less metrics and there isn't a mechanism to speed up or slow down the interval dynamically.
So I just thought, why not stream the metrics?
But `[[inputs.http]]` insists on doing a gather-run per interval and I don't see how to switch it to streaming like `[[inputs.execd]]` can do.
So... Feature Request: Streaming with `[[inputs.http]]`, or maybe you have an alternative for me?
### Expected behavior
If the connection doesn't close (but in a sound way, like chunked streaming) then don't complain about timeouts but read the streamed data.
### Actual behavior
data streaming not possible.
### Additional info
_No response_
Contributor guide
Research direction
Start by locating the inputs.http implementation and its gather, timeout, and response-reading paths. Trace how it handles connections that remain open, then determine the boundaries of supporting chunked streaming without treating the connection as a timeout. Done means streamed metrics can be read continuously while preserving the expected inputs.http behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend, networking
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100