Graylog2 / Graylog2/graylog2-server

Remove obsolete migrations related to 4.x and prior

Open
#23,860 0 comments 0 reactions 0 assignees View on GitHub
improvement triaged
Dominant language
Java
Stars
8.1k
Forks
1.1k
Avg merge
1d 20h
Merged PRs (30d)
217

Description

With 4.x being EOL it's a good opportunity to clean up the ever-growing set of migrations. This issue examines migrations through 2022.

**Obsolete Migrations:**

- `V2018070614390000_EnforceUniqueGrokPatterns` adds a unique index for grok patterns. All supported versions now have this index and we no longer need to run the migration.
- `V20161215163900_MoveIndexSetDefaultConfig` removes the `default` field from the `IndexSetConfig` database object and creates a `DefaultIndexSetConfig` object in the cluster config instead.
There should be no instances that still have this obsolete configuration. `V20161116172100_DefaultIndexSetMigration` ensures there is a valid cluster config entry.
- `V20170110150100_FixAlertConditionsMigration` During 2.2 development the values for integer parameters were stored as strings in MongoDB. This migration fixes any existing broken documents. No such documents should still exist.
- `V2018070614390000_EnforceUniqueGrokPatterns` no longer needed since we have a unique index
- `V20191219090834_AddSourcesPage` already modified to do nothing
- `V20211221144300_GeoIpResolverConfigMigration` no instances of the old schema should still exist

**Migrations that should be replaced by a simplified / modernized version:**

- `V20151210140600_ElasticsearchConfigMigration` and `V20161116172100_DefaultIndexSetMigration` (see #14820)
- `V20180924111644_AddDefaultGrokPatterns` and `V20191121145100_FixDefaultGrokPatterns` can be combined - no need to continue installing obsolete patterns and then fixing them. There are no instances that still have the obsolete patterns.

**Unclear - needs research**

- `V20220623125450_AddJobTypeToJobTrigger`

**Old migrations that are still needed**
Primarily, these are migrations that populate the DB when starting with a blank DB. They also provide a - very limited - way of restoring a corrupted instance to a valid state.
We should also retain migrations required to upgrade from 4.x to 5.0

- `V19700101000000_GenerateClusterId` setup
- `V20151210140600_AddSearchesClusterConfigMigration` setup
- `V20161116172200_CreateDefaultStreamMigration` setup
- `V20161122174500_AssignIndexSetsToStreamsMigration` setup
- `V20170607164210_MigrateReopenedIndicesToAliases` required for upgrade to 5.0
- `V20180718155800_AddContentPackIdAndRev` we can't control content packs as tightly
- `V20180924111644_AddDefaultGrokPatterns` setup
- `V20190705071400_AddEventIndexSetsMigration(ElasticsearchConfiguration elasticsearchConfiguration` setup
- `V20190730000000_CreateDefaultEventsConfiguration(EventsConfigurationProvider configProvider` setup
- `V20190730100900_AddAlertsManagerRole` setup
- `V20191129134600_CreateInitialUrlAllowlist` setup
- `V20200102140000_UnifyEventSeriesId` needed for back-compat of event definitions in content packs
- `V20200226181600_EncryptAccessTokensMigration` keep in case we want to change the cipher
- `V20200722110800_AddBuiltinRoles` setup
- `V20201103145400_LegacyAuthServiceMigration` this is needed in pluggable auth service migrations
- `V20220719130704_ImprovedDefaultProcessingOrderMigration` setup
- `V20220818112023_AddStreamMatcherToProcessingOrderMigration` setup

## Context

- Reduces startup time
- Reduces amount of code needing to be maintained

## Your Environment

* Graylog Version: 7.0

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.