elastic / elastic/go-docappender

Handling for 413 entity too large in BulkIndexer

Open
#210 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
11
Forks
20
PR merge metrics
No merged PRs in 30d

Description

While investigating https://github.com/elastic/ingest-dev/issues/3677, I noticed that this package does not handle a 413 Entity Too Large status from Elasticsearch. Even with batching, in specific use cases like in Beats (which may include large stack traces in documents), a batch can still exceed http.max_content_length, the maximum request size in Elasticsearch, which defaults to 100MB.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing how BulkIndexer sends batches and handles Elasticsearch responses. Reproduce or inspect the 413 Entity Too Large case described in the issue, then define and verify the expected handling for batches exceeding Elasticsearch's http.max_content_length.

Written by the indexing model from the issue text.

Assessment

Tech stack
elasticsearch, go
Domain
backend, search
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.