elastic / elastic/logstash

logstash gc cost much time and too many 429.

Open
#17,799 0 comments 0 reactions 0 assignees View on GitHub
enhancement status:needs-triage
Dominant language
Java
Stars
14.9k
Forks
3.5k
Avg merge
1d 4h
Merged PRs (30d)
88

Description

Image

when the fluent-bit send lots of log.

the logstash would return many 429.

i think it is the gc problem.
Image

#### logstash version
logstash 8.15, jdk 21, g1 gc

-Xms Xmx 6G

memory limit 12GB.

Contributor guide

Open the contributing guide

Research direction

No source files, tests, or entry points are named. Start by reproducing the issue with Fluent Bit sending a high log volume to Logstash 8.15 using JDK 21, G1 GC, and the stated memory limits; correlate GC pauses with 429 responses. Done means the cause of the 429 responses is established and the behavior is validated under the reported workload.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.