elastic / elastic/go-docappender

Appender does not flush documents which received 429 if bulk indexer is never active

Open
#148 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
11
Forks
20
PR merge metrics
No merged PRs in 30d

Description

Appender does not handle this edge case well. e.g. Appender receives event E1. Bulk Indexer BI1 handles E1, but gets a document level 429. This document is then rewritten to BI1's buffer. But consider a scenario where BI1 is never scheduled to be active, then E1 will sit inside BI1's buffer forever, past flush timeout, until Appender.Close is called.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing Appender and Bulk Indexer handling for a document-level 429, focusing on how E1 is rewritten to BI1's buffer and how the flush timeout is applied. Reproduce the case where BI1 is never scheduled, then verify that the buffered document is flushed before Appender.Close.

Written by the indexing model from the issue text.

Assessment

Tech stack
elasticsearch, go
Domain
backend, databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.