Graylog2 / Graylog2/graylog2-server

Multiple Plugin Directories

Open
#6,482 0 comments 1 reaction 0 assignees View on GitHub
feature triaged
Dominant language
Java
Stars
8.1k
Forks
1.1k
Avg merge
1d 20h
Merged PRs (30d)
217

Description

Since Graylog ships builtin plugins in `/usr/share/graylog/plugin` directory within the Docker image, users are required to mount multiple volumes into this directory when including third-party plugins. While this process works it's not scalable to all environments, Kubernetes specifically. Inclusion of the default plugins has benefits however, users can no longer change the default directory with environment variable`GRAYLOG_PLUGIN_DIR` without impacting the default plugins.

## Expected Behavior

When using Kubernetes you're unable to mount jar files into the plugins directory without overwriting the existing default plugins included in the image. One possible option would be to simply allow multiple directories to be set in the `GRAYLOG_PLUGIN_DIR` environment variable. When the variable is defined Graylog would load all plugins found within any directory provided. Allowing this configuration will allow users to supply an additional directory with plugins without impacting the defaults.

Example: `GRAYLOG_PLUGIN_DIR=/usr/share/graylog/plugin:/var/lib/graylog/plugins`

## Current Behavior

Today when setting `GRAYLOG_PLUGIN_DIR` to a directory other than the default plugins directory all included plugins are broken. This creates a number of errors within the logs around broken adapters.

```
2019-09-17 17:11:34,479 WARN : org.graylog2.lookup.LookupTableService - Unable to load data adapter whois of type whois, missing a factory. Is a required plugin missing?
2019-09-17 17:11:34,479 WARN : org.graylog2.lookup.LookupTableService - Unable to load data adapter otx-api-domain of type otx-api, missing a factory. Is a required plugin missing?
2019-09-17 17:11:34,480 WARN : org.graylog2.lookup.LookupTableService - Unable to load data adapter otx-api-ip of type otx-api, missing a factory. Is a required plugin missing?

2019-09-17 17:11:34,708 WARN : org.graylog2.lookup.LookupTableService - Lookup table whois is referencing a missing data adapter 5cef4db5146bc60012abe85e, check if it started properly.
2019-09-17 17:11:34,708 WARN : org.graylog2.lookup.LookupTableService - Lookup table otx-api-ip is referencing a missing data adapter 5cef4ded3f0a1c00129474e5, check if it started properly.
2019-09-17 17:11:34,708 WARN : org.graylog2.lookup.LookupTableService - Lookup table otx-api-domain is referencing a missing data adapter 5cef4ded3f0a1c00129474e1, check if it started properly.
```

## Possible Solution

Allow for environment variable `GRAYLOG_PLUGIN_DIR` to handle multiple directories.
Example: `GRAYLOG_PLUGIN_DIR=/usr/share/graylog/plugin:/var/lib/graylog/plugins`

## Context

I changed the default directory so I could manage plugins in Kubernetes on persistent volumes. Once changed included plugins no longer work. Conversation on the forms can be found [here](https://community.graylog.org/t/missing-data-adapters-and-plugins/11995).

## Your Environment

* Graylog Version: `3.1.2`
* Elasticsearch Version: `6.7.2`
* MongoDB Version: `4.0`
* Operating System: Graylog Docker images on Kubernetes
* Browser version: Chrome, Firefox, etc.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.