processor/stream: server blocks on stream end when batch size < 10
- Dominant language
- Go
- Stars
- 1.3k
- Forks
- 543
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 109
Description
The `processor/stream` code reads in batches of 10 events at a time:
https://github.com/elastic/apm-server/blob/17e0f7a9f1a3aa1453e5df8b7d4cf746e9d7179d/processor/stream/processor.go#L279
Once a batch is received, they are dispatched to the publisher, which transforms and sends them through the libbeat pipeline to be recorded in Elasticsearch.
By default, agents will close the stream after 10 seconds, or after it reaches a certain size (~750K). So if an agent sends fewer than 10 events, the `processor/stream` code will generally block waiting for the stream to end before it dispatches to the publisher.
We should consider adding a timeout (or context with timeout) to the `StreamReader.Read` method to avoid this.
Contributor guide
Assessment
This issue has not been assessed yet.