elastic / elastic/logstash

Logstash 6.0 allows starting with the -r flag with plugins that don't support reloading

Open
#8,021 7 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
14.9k
Forks
3.5k
Avg merge
1d 4h
Merged PRs (30d)
88

Description

With the following configuration: `input { stdin { }} output { stdout { codec => rubydebug } }`

Logstash 5.5.0 prevents you from starting with the `-r` flag with plugins without auto-reloading support:
```
logstash-5.5.0 $ logstash_stdin -r
ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console.
Sending Logstash's logs to /Users/jake/workspace/releases/logstash-5.5.0/logs which is now configured via log4j2.properties
[2017-08-17T18:41:32,922][ERROR][logstash.agent ] Logstash is not able to start since configuration auto reloading was enabled but the configuration contains plugins that don't support it. Quitting... {:pipeline_id=>"main", :plugins=>[LogStash::Inputs::Stdin]}
```

Logstash 6.0.0 allows you to restart with plugin's that are not supported:
```
logstash-6.0.0-beta1 $ logstash_stdin -r
Sending Logstash's logs to /Users/jake/workspace/releases/logstash-6.0.0-beta1/logs which is now configured via log4j2.properties
[2017-08-17T18:32:07,730][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2017-08-17T18:32:07,950][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
[2017-08-17T18:32:09,018][INFO ][logstash.pipeline ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>1000, :thread=>"#"}
[2017-08-17T18:32:14,035][INFO ][logstash.pipeline ] Pipeline started {"pipeline.id"=>"main"}
The stdin plugin is now waiting for input:
[2017-08-17T18:32:14,045][INFO ][logstash.agent ] Pipelines running {:count=>1, :pipelines=>["main"]}
[2017-08-17T18:32:35,053][ERROR][logstash.agent ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Cannot reload pipeline, because the existing pipeline is not reloadable"}
[2017-08-17T18:32:38,048][ERROR][logstash.agent ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Cannot reload pipeline, because the existing pipeline is not reloadable"}
[2017-08-17T18:32:41,048][ERROR][logstash.agent ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Cannot reload pipeline, because the existing pipeline is not reloadable"}
[2017-08-17T18:32:44,051][ERROR][logstash.agent ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Cannot reload pipeline, because the existing pipeline is not reloadable"}
[2017-08-17T18:32:47,053][ERROR][logstash.agent ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Cannot reload pipeline, because the existing pipeline is not reloadable"}
[2017-08-17T18:32:50,055][ERROR][logstash.agent ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Cannot reload pipeline, because the existing pipeline is not reloadable"}
```
Both version are using `logstash-input-stdin-3.2.3`

Contributor guide

Open the contributing guide

Research direction

Reproduce the issue with the shown stdin and stdout configuration by starting Logstash with the -r flag, then trace the configuration auto-reloading path through the Logstash agent and pipeline behavior described in the logs. Compare the 5.5.0 rejection with the 6.0.0 behavior and add coverage where the existing tests for reloadability are located. Done means startup rejects a non-reloadable plugin configuration instead of repeatedly reporting that the pipeline cannot reload.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.