Graylog2 / Graylog2/graylog2-server

Data Node Migration finished with not messages migrated.

Open
#18,695 0 comments 0 reactions 0 assignees View on GitHub
bug data-node test-day to-verify
Dominant language
Java
Stars
8.1k
Forks
1.1k
Avg merge
1d 20h
Merged PRs (30d)
217

Description

## In the process of re-verification

## Expected Behavior
Messages are migrated.

## Current Behavior
Messages are not migrated, data-node looks like a fresh search engine after migration. No error in migration UI.

Data-node after migration:
![image](https://github.com/Graylog2/graylog2-server/assets/100699120/a7c347c3-f0c0-4aab-961d-2cd1e588b56c)

OS before migration:
```
yellow open security-auditlog-2024.03.19 xc4Zf9ZrSUK-N4BtWde9NQ 1 1 420 0 933.4kb 933.4kb
green open .plugins-ml-config L4EH3H5lQiSXVwoyB9u05w 1 0 1 0 3.9kb 3.9kb
green open .opensearch-observability oWRdnh2sQ82696Z3wl0EEw 1 0 0 0 208b 208b
green open gl-failures_0 rwpC-sHGRgKUz8dvwOhlGA 1 0 0 0 208b 208b
green open investigation_message_index_0 8AkCXczARWW70Dr8TDxLVw 1 0 0 0 208b 208b
green open graylog_0 BEQMTOAYSoSqBwQ-LSIT3w 1 0 25751 0 6.8mb 6.8mb
green open gl-events_0 doEpLX0HRFCviZyM7XYG_w 1 0 0 0 208b 208b
green open gl-system-events_0 CeuR9L__SBewsMvnyK9BtA 1 0 1 0 10kb 10kb
green open investigation_event_index_0 anAkLE-MQliGSoDBnh5VmQ 1 0 0 0 208b 208b
green open .opendistro_security 8Y2VFJOKQPCOjdG8oNVfyA 1 0 10 0 77.7kb 77.7kb
green open asd_0 GeUt1KTeTZmXomJLPiEkyg 1 0 0 0 208b 208b
```

Around 25k messages seems to have not been migrated...

## Possible Solution
I was not even able to find out why this had happened...

Humble suggestion: maybe we should store some statistics of indices before migration and compare it to the statistics after migration, so that we have a clue it it really happened?

## Steps to Reproduce (for bugs)
1. Download, install, start OS locally.
2. Run GL.
3. Run data-node, configure it via preflight UI.
4. Set `opensearch_data_location=/var/lib/opensearch/`
5. Go through migrations steps and follow instructions.

## Your Environment
I've set data-node to use different ports than OS, to not get fooled what is what.
```
opensearch_http_port = 9201
opensearch_transport_port = 9301

Requests to data node via REST API browser work correctly (see the previous screenshot), direct calls from browser (i.e. https://lkaminski-dell:9201/_cat) produce `Authentication finally failed` response...

* Graylog Version: 6.0-beta2
* OpenSearch Version: 2.12
* Operating System: Linux

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.