elastic / elastic/fleet-server

Limit ES bulker request body size

Open
#2,738 0 comments 0 reactions 0 assignees View on GitHub
enhancement Team:Fleet
Dominant language
Go
Stars
113
Forks
117
Avg merge
1d 16h
Merged PRs (30d)
112

Description

**Describe the enhancement:**

We need a way to limit the size of bulk requests sent to Elasticsearch.
Currently if the request size gets too large (where ES will return an error), fleet-server will not clear its bulk queue (the bulker only resets the queue on [successful requests](https://github.com/elastic/fleet-server/blob/main/internal/pkg/bulk/engine.go#L182-L189)) and all subsequent bulk requests will fail.

The default size should be the same as bulk request size for elasticsearch, currently the [request size limit is 100mb](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html#docs-bulk-api-desc).
This value should be controlled through config as well.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.