influxdata / influxdata/influxdb
InfluxDB 3 rejects Telegraf elasticsearch_http metrics with very long lines (>200k chars)
- Dominant language
- Rust
- Stars
- 31.7k
- Forks
- 3.7k
- Avg merge
- 13h 37m
- Merged PRs (30d)
- 8
Description
__Steps to reproduce:__
Telegraf configuration
```
[[inputs.elasticsearch]]
servers = ["http://elastic:token@host:9200"]
```
Run Telegraf
`telegraf --once --debug --input-filter elasticsearch`
Write target: InfluxDB 3 /api/v2/write
__Expected behaviour:__
All line protocol generated by the official Telegraf Elasticsearch plugin should be accepted by InfluxDB 3 if it is already accepted by InfluxDB 2.
__Actual behaviour:__
Metrics of measurement:
`elasticsearch_http`
are rejected by InfluxDB 3 with:
`parsing for line protocol failed`
The identical payload is accepted by InfluxDB 2.
__Environment info:__
* InfluxDB version: InfluxDB 3 Core 3.8.0
* Endpoint used: /api/v2/write
* Telegraf version: 1.37.1
* Operating system: SLES Linux
* Output of uname -srm: Linux 6.4.0-150700.53.28-default x86_64
* Running InfluxDB 3 in a containerized environment (Docker)
__Config:__
```
[[inputs.elasticsearch]]
servers = ["http://elastic:token@host:9200"]
http_timeout = "5s"
local = true
cluster_health = true
cluster_health_level = "indices"
cluster_stats = true
cluster_stats_only_from_master = true
indices_include = ["_all"]
indices_level = "shards"
```
__Logs:__
```
Error while handling request error=write buffer error:
parsing for line protocol failed method=POST path=/api/v2/write
```
__Investigation Result:__
By sending lines one-by-one to InfluxDB 3, we identified that a single line of measurement elasticsearch_http with a length of approximately 200,000+ characters causes the failure.
Dropping this measurement in Telegraf resolves the issue completely:
`namedrop = ["elasticsearch_http"]`
This strongly suggests a parser limitation in InfluxDB 3 regarding maximum line length or field count.
Contributor guide
Research direction
The failing entry point is the /api/v2/write endpoint; reproduce with Telegraf's elasticsearch input and a single elasticsearch_http line over 200,000 characters. Compare behavior with InfluxDB 2, and consider the issue resolved when the same payload is accepted by InfluxDB 3 without the parsing error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elasticsearch, rust
- Domain
- api, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100