influxdata / influxdata/telegraf

FR: inputs.http could support streaming with chunked encoding

Open
#18,414 2 comments 0 reactions 0 assignees View on GitHub
feature request
Dominant language
Go
Stars
17.8k
Forks
5.8k
Avg merge
1d 20h
Merged PRs (30d)
161

Description

### Use Case

I currently try to stream metrics in influx line protocol from one telegraf to the next. The receiving telegraf opens the connection to a python-script-webserver (outputs.execd) with bearer token authentication.

This all works well and good, but I have to batch the data and the requesting telegraf is tied to an interval in which there might be more or less metrics and there isn't a mechanism to speed up or slow down the interval dynamically.
So I just thought, why not stream the metrics?

But `[[inputs.http]]` insists on doing a gather-run per interval and I don't see how to switch it to streaming like `[[inputs.execd]]` can do.

So... Feature Request: Streaming with `[[inputs.http]]`, or maybe you have an alternative for me?

### Expected behavior

If the connection doesn't close (but in a sound way, like chunked streaming) then don't complain about timeouts but read the streamed data.

### Actual behavior

data streaming not possible.

### Additional info

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by locating the inputs.http implementation and its gather, timeout, and response-reading paths. Trace how it handles connections that remain open, then determine the boundaries of supporting chunked streaming without treating the connection as a timeout. Done means streamed metrics can be read continuously while preserving the expected inputs.http behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend, networking
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.