Asynchronous start of input plugins
- Dominant language
- Java
- Stars
- 14.9k
- Forks
- 3.5k
- Avg merge
- 19h 14m
- Merged PRs (30d)
- 63
Description
This line of inquiry was inspired by the question here: https://github.com/logstash-plugins/logstash-input-rabbitmq/issues/58
This is how it works today, which is great, but has a problem I will describe below.
1. Logstash connects to rabbitmq server starts ingesting logs
2. RabbitMQ Goes down, logstash repeatedly retries connecting
3. Connection attempt succeeds, logstash succeeds in processing logs.
The above is great, however, if in step 2. Logstash is restarted, ALL inputs are blocked until the one RabbitMQ input connected to the down server is connected since its `register` method blocks waiting for an RMQ server.
I propose the following:
1. Input plugins should have their register method invoked asynchronously
2. Input plugins should have an internal state machine with the states `:starting, :running, :stopped, :permanent_error, :transient_error`. These states could be accessible via API, and state transitions would be logged.
These state machines could be really useful from an operator standpoint if exposed from an API or used in a centralized config mgmt system.
Contributor guide
Assessment
This issue has not been assessed yet.