Graylog2 / Graylog2/graylog2-server
Multiple Plugin Directories
- Dominant language
- Java
- Stars
- 8.1k
- Forks
- 1.1k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 217
Description
Since Graylog ships builtin plugins in `/usr/share/graylog/plugin` directory within the Docker image, users are required to mount multiple volumes into this directory when including third-party plugins. While this process works it's not scalable to all environments, Kubernetes specifically. Inclusion of the default plugins has benefits however, users can no longer change the default directory with environment variable`GRAYLOG_PLUGIN_DIR` without impacting the default plugins.
## Expected Behavior
When using Kubernetes you're unable to mount jar files into the plugins directory without overwriting the existing default plugins included in the image. One possible option would be to simply allow multiple directories to be set in the `GRAYLOG_PLUGIN_DIR` environment variable. When the variable is defined Graylog would load all plugins found within any directory provided. Allowing this configuration will allow users to supply an additional directory with plugins without impacting the defaults.
Example: `GRAYLOG_PLUGIN_DIR=/usr/share/graylog/plugin:/var/lib/graylog/plugins`
## Current Behavior
Today when setting `GRAYLOG_PLUGIN_DIR` to a directory other than the default plugins directory all included plugins are broken. This creates a number of errors within the logs around broken adapters.
```
2019-09-17 17:11:34,479 WARN : org.graylog2.lookup.LookupTableService - Unable to load data adapter whois of type whois, missing a factory. Is a required plugin missing?
2019-09-17 17:11:34,479 WARN : org.graylog2.lookup.LookupTableService - Unable to load data adapter otx-api-domain of type otx-api, missing a factory. Is a required plugin missing?
2019-09-17 17:11:34,480 WARN : org.graylog2.lookup.LookupTableService - Unable to load data adapter otx-api-ip of type otx-api, missing a factory. Is a required plugin missing?
2019-09-17 17:11:34,708 WARN : org.graylog2.lookup.LookupTableService - Lookup table whois is referencing a missing data adapter 5cef4db5146bc60012abe85e, check if it started properly.
2019-09-17 17:11:34,708 WARN : org.graylog2.lookup.LookupTableService - Lookup table otx-api-ip is referencing a missing data adapter 5cef4ded3f0a1c00129474e5, check if it started properly.
2019-09-17 17:11:34,708 WARN : org.graylog2.lookup.LookupTableService - Lookup table otx-api-domain is referencing a missing data adapter 5cef4ded3f0a1c00129474e1, check if it started properly.
```
## Possible Solution
Allow for environment variable `GRAYLOG_PLUGIN_DIR` to handle multiple directories.
Example: `GRAYLOG_PLUGIN_DIR=/usr/share/graylog/plugin:/var/lib/graylog/plugins`
## Context
I changed the default directory so I could manage plugins in Kubernetes on persistent volumes. Once changed included plugins no longer work. Conversation on the forms can be found [here](https://community.graylog.org/t/missing-data-adapters-and-plugins/11995).
## Your Environment
* Graylog Version: `3.1.2`
* Elasticsearch Version: `6.7.2`
* MongoDB Version: `4.0`
* Operating System: Graylog Docker images on Kubernetes
* Browser version: Chrome, Firefox, etc.
Contributor guide
Assessment
This issue has not been assessed yet.