Track shutdown refactor for other plugins not packaged with logstash
- Dominant language
- Java
- Stars
- 14.9k
- Forks
- 3.5k
- Avg merge
- 19h 14m
- Merged PRs (30d)
- 63
Description
A follow up to https://github.com/elastic/logstash/issues/3813. This issue focuses on plugins that are not listed in https://github.com/elastic/logstash/issues/3813
#### Recap of refactor task
1. add `stop` (public), `stop?` (public) to LogStash::Inputs::Base
2. rename `teardown` to `close`
3. remove `shutdown`, `finished`. `finished?`, `running?`, `terminating?` from LogStash::Plugin
## Refactor work needed on the plugins
#### [Input plugins](https://github.com/issues?utf8=%E2%9C%93&q=is%3Aopen+is%3Aissue+user%3Alogstash-plugins++label%3A%22shutdown+semantics%22)
Input plugins are now shutdown by an external call to `plugin.stop` instead of catching LogStash::ShutdownSignal exception.
Unless overridden, `stop` will simply makes `stop?` return `true`, thus allowing `run` to poll this and return after seeing the change.
In some plugins extra work must be done in `stop` to instruct `run` that it's time to return. For example, in the `logstash-input-udp` it's necessary to call `@socket.close` to make the blocking read on the socket raise an exception, thus breaking out the loop.
So, different input plugins will require different `stop` strategies.
Refactoring an input plugin involves:
- removing rescue of `ShutdownSignal` exception
- understand the nature of the `run` loop: is it a `while`? a `consumer.subscribe {|event| }`? is there a blocking operation on a socket/fd?
- use `stop?` and/or override `stop` to make run return
- put any other cleanup/bookkeeping tasks in `close` (currently done in `teardown`)
- remove any calls to `shutdown`, `finished`, `finished?`, `running?` or `terminating?`
Then for testing you can use the shared example provided in logstash-devutils:
``` ruby
describe LogStash::Inputs::Http do
let(:port) { rand(5000) + 1025 }
it_behaves_like "an interruptible input plugin" do
let(:config) { { "port" => port } }
end
end
```
##### Input plugin refactor
- [x] [logstash-input-drupal_dblog](https://github.com/logstash-plugins/logstash-input-drupal_dblog/issues/10) PR: https://github.com/logstash-plugins/logstash-input-drupal_dblog/pull/11
- [ ] [logstash-input-dynamodb](https://github.com/logstash-plugins/logstash-input-dynamodb/issues/3)
- [ ] [logstash-input-example](https://github.com/logstash-plugins/logstash-input-example/issues/4)
- [ ] [logstash-input-fluentd](https://github.com/logstash-plugins/logstash-input-fluentd/issues/1)
- [ ] [logstash-input-gemfire](https://github.com/logstash-plugins/logstash-input-gemfire/issues/2)
- [ ] [logstash-input-github](https://github.com/logstash-plugins/logstash-input-github/issues/7)
- [ ] [logstash-input-heroku](https://github.com/logstash-plugins/logstash-input-heroku/issues/7)
- [x] [logstash-input-http_poller](https://github.com/logstash-plugins/logstash-input-http_poller/issues/36)
- [x] [logstash-input-jdbc](https://github.com/logstash-plugins/logstash-input-jdbc/issues/52) PR: https://github.com/logstash-plugins/logstash-input-jdbc/pull/53
- [ ] [logstash-input-jms](https://github.com/logstash-plugins/logstash-input-jms/issues/6)
- [ ] [logstash-input-jmx](https://github.com/logstash-plugins/logstash-input-jmx/issues/11)
- [ ] [logstash-input-log4j2](https://github.com/logstash-plugins/logstash-input-log4j2/issues/1)
- [ ] [logstash-input-meetup](https://github.com/logstash-plugins/logstash-input-meetup/issues/5)
- [x] [logstash-input-neo4j](https://github.com/logstash-plugins/logstash-input-neo4j/issues/3) PR https://github.com/logstash-plugins/logstash-input-neo4j/pull/4
- [ ] [logstash-input-perfmon](https://github.com/logstash-plugins/logstash-input-perfmon/issues/2)
- [ ] [logstash-input-puppet_facter](https://github.com/logstash-plugins/logstash-input-puppet_facter/issues/5)
- [ ] [logstash-input-rackspace](https://github.com/logstash-plugins/logstash-input-rackspace/issues/3)
- [x] [logstash-input-relp](https://github.com/logstash-plugins/logstash-input-relp/issues/10)
- [ ] [logstash-input-rss](https://github.com/logstash-plugins/logstash-input-rss/issues/10)
- [ ] [logstash-input-salesforce](https://github.com/logstash-plugins/logstash-input-salesforce/issues/3)
- [x] [logstash-input-sqlite](https://github.com/logstash-plugins/logstash-input-sqlite/issues/3)
- [x] [logstash-input-stomp](https://github.com/logstash-plugins/logstash-input-stomp/issues/5)
- [ ] [logstash-input-varnishlog](https://github.com/logstash-plugins/logstash-input-varnishlog/issues/3)
- [ ] [logstash-input-websocket](https://github.com/logstash-plugins/logstash-input-websocket/issues/4)
- [ ] [logstash-input-wmi](https://github.com/logstash-plugins/logstash-input-wmi/issues/10) PR https://github.com/logstash-plugins/logstash-input-wmi/pull/11
- [x] [logstash-input-zenoss](https://github.com/logstash-plugins/logstash-input-zenoss/issues/3)
##### 2. Remove `terminating?`
In a lot of scenarios the call to `terminating?` might be replaced with `stop?` in the case of input plugins
- [x] [logstash-output-gemfire](https://github.com/logstash-plugins/logstash-output-gemfire/blob/7a2ab3f2dd87ab05011f7affe53261d95933ff67/lib/logstash/outputs/gemfire.rb#L64) PR: https://github.com/logstash-plugins/logstash-output-gemfire/pull/4
##### 3. Remove `finished`
- [x] [logstash-output-cloudwatch](https://github.com/logstash-plugins/logstash-output-cloudwatch) PR: https://github.com/logstash-plugins/logstash-output-cloudwatch/pull/5
- [x] [logstash-output-gemfire](https://github.com/logstash-plugins/logstash-output-gemfire) PR: https://github.com/logstash-plugins/logstash-output-gemfire/pull/4
- [x] [logstash-output-jira](https://github.com/logstash-plugins/logstash-output-jira): PR: https://github.com/logstash-plugins/logstash-output-jira/pull/6
- [x] [logstash-output-loggly](https://github.com/logstash-plugins/logstash-output-loggly) PR: https://github.com/logstash-plugins/logstash-output-loggly/pull/10
- [x] [logstash-output-nagios_nsca](https://github.com/logstash-plugins/logstash-output-nagios_nsca): finished PR: https://github.com/logstash-plugins/logstash-output-nagios_nsca/pull/6
- [x] [logstash-output-redmine](https://github.com/logstash-plugins/logstash-output-redmine): PLEASE REVIEW https://github.com/logstash-plugins/logstash-output-redmine/pull/3
#### 7. Rename `teardown` to `close`
- [x] [logstash-codec-compress_spooler](https://github.com/logstash-plugins/logstash-codec-compress_spooler)
- [x] [logstash-codec-spool](https://github.com/logstash-plugins/logstash-codec-spool)
- [x] [logstash-input-couchdb_changes](https://github.com/logstash-plugins/logstash-input-couchdb_changes)
- [x] [logstash-input-gemfire](https://github.com/logstash-plugins/logstash-input-gemfire)
- [ ] [logstash-input-github](https://github.com/logstash-plugins/logstash-input-github)
- [x] [logstash-input-jmx](https://github.com/logstash-plugins/logstash-input-jmx)
- [x] [logstash-input-neo4j](https://github.com/logstash-plugins/logstash-input-neo4j)
- [x] [logstash-input-rackspace](https://github.com/logstash-plugins/logstash-input-rackspace)
- [x] [logstash-input-relp](https://github.com/logstash-plugins/logstash-input-relp)
- [x] [logstash-input-varnishlog](https://github.com/logstash-plugins/logstash-input-varnishlog)
- [x] [logstash-output-gemfire](https://github.com/logstash-plugins/logstash-output-gemfire)
- [x] [logstash-output-google_bigquery](https://github.com/logstash-plugins/logstash-output-google_bigquery)
- [x] [logstash-output-google_cloud_storage](https://github.com/logstash-plugins/logstash-output-google_cloud_storage)
- [x] [logstash-output-influxdb](https://github.com/logstash-plugins/logstash-output-influxdb) PR: https://github.com/logstash-plugins/logstash-output-influxdb/pull/28/files
- [x] [logstash-output-jms](https://github.com/logstash-plugins/logstash-output-jms) PR: https://github.com/logstash-plugins/logstash-output-jms/pull/2
- [x] [logstash-output-juggernaut](https://github.com/logstash-plugins/logstash-output-juggernaut)
- [ ] [logstash-output-neo4j](https://github.com/logstash-plugins/logstash-output-neo4j)
- [x] [logstash-output-solr_http](https://github.com/logstash-plugins/logstash-output-solr_http)
- [x] [logstash-output-statsd](https://github.com/logstash-plugins/logstash-output-statsd)
- [x] [logstash-output-webhdfs](https://github.com/logstash-plugins/logstash-output-webhdfs)
Contributor guide
Assessment
This issue has not been assessed yet.