Graylog2 / Graylog2/graylog2-server

Detect if migrations have been executed

Open
#16,234 2 comments 0 reactions 0 assignees View on GitHub
feature triaged
Dominant language
Java
Stars
8.1k
Forks
1.1k
Avg merge
1d 20h
Merged PRs (30d)
217

Description

A Graylog cluster only works correctly, if all MongoDB migrations have been executed. There are currently two options how migrations are run:
1. The leader node of a cluster executes migrations in a dedicated startup phase
2. Migrations are explicitly triggered by running the `graylog migrate` command prior to starting any node in the cluster

If a node is started without migrations being run, this is potentially dangerous. This applies to both a pristine Graylog setup or, more importantly after a version upgrade of a Graylog cluster.

There are a couple of scenarios how this could happen:
1. A non-leader node is started as the first node.
2. Automatic execution of migrations has been disabled (`run_migrations=false`) but the `graylog migrate` command has not been executed prior to starting a node

We should detect this situation and handle it properly, e.g. by delaying node startup until migrations have been executed by a different node or the migration command. We might also abort node startup altogether.

As a detection mechanism we could keep track of the set of migrations that have been executed in MongoDB. Alternatively we could track the software version of the cluster in MongoDB and update it only after migrations have been run successfully. There might be other options.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.