elastic / elastic/logstash

reversing the launch order of plugins in the pipeline

Open
#4,092 1 comment 0 reactions 0 assignees View on GitHub
discuss
Dominant language
Java
Stars
14.9k
Forks
3.5k
Avg merge
19h 14m
Merged PRs (30d)
63

Description

Looking at the pipeline run method we can see:

``` ruby
begin
start_inputs
start_filters if filters?
start_outputs
ensure
# it is important to garantee @ready to be true after the startup sequence has been completed
# to potentially unblock the shutdown method which may be waiting on @ready to proceed
@ready.make_true
end
```

What if the start up order was reversed? Like this:

``` ruby
begin
start_outputs
start_filters if filters?
start_inputs
ensure
# it is important to garantee @ready to be true after the startup sequence has been completed
# to potentially unblock the shutdown method which may be waiting on @ready to proceed
@ready.make_true
end
```

Inputs should be the last thing to be started in the pipeline. This has a very nice property: if a problem with a consumer can be detected at `output.register` time, then we can immediately terminate logstash before it starts receiving events.

Currently, logstash can receive an event, and in the meantime some output plugin will raise an exception during `#register`, but logstash now has inflight events :bomb:

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.