Multiple logstash getting input from 1 Redis
- Dominant language
- Java
- Stars
- 14.9k
- Forks
- 3.5k
- Avg merge
- 19h 14m
- Merged PRs (30d)
- 63
Description
Hello,
I am running into the following behavior. I have 2 logstash indexers reading from one redis server. I've done about a dozen of test runs by sending 20K log lines each time, and when I look in Kibana, I always see one logstash server processing at least twice as many log events. Is there any way to adjust so they balance each other out when reading from one redis queue? I tried changing the batch_count to 100, but still issue.
For all general issues, please provide the following details for fast resolution:
- Version:
- Operating System:
- Config File (if you have sensitive info, please remove it):
```
input {
redis {
host => '10.218.249.33'
data_type => 'list'
key => 'filebeat'
}
}
filter {
mutate{
add_field => {
"logstash_server" => "server-1"
}
}
}
output {
stdout{ codec => rubydebug}
amazon_es {
hosts => ["aws-cluster-123.us-west-2.es.amazonaws.com"]
region => "us-west-2"
index => "logstash-webserver-tcast-%{+YYYY.MM.dd}"
}
}
```
- Sample Data:
- Steps to Reproduce:
Contributor guide
Assessment
This issue has not been assessed yet.