elastic / elastic/apm-server

processor/stream: server blocks on stream end when batch size < 10

Open
#3,265 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Go
Stars
1.3k
Forks
543
Avg merge
1d 18h
Merged PRs (30d)
109

Description

The `processor/stream` code reads in batches of 10 events at a time:

https://github.com/elastic/apm-server/blob/17e0f7a9f1a3aa1453e5df8b7d4cf746e9d7179d/processor/stream/processor.go#L279

Once a batch is received, they are dispatched to the publisher, which transforms and sends them through the libbeat pipeline to be recorded in Elasticsearch.

By default, agents will close the stream after 10 seconds, or after it reaches a certain size (~750K). So if an agent sends fewer than 10 events, the `processor/stream` code will generally block waiting for the stream to end before it dispatches to the publisher.

We should consider adding a timeout (or context with timeout) to the `StreamReader.Read` method to avoid this.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.