graylog-datanode (OpenSearch) in Crash Loop due to hard-coded -Djava.security.manager=allow

Hello,

I am experiencing a critical issue where the graylog-datanode service fails to start and enters a restart loop. I have performed extensive troubleshooting and believe this is a bug in the package itself. (NOTE: I am a newbie and used Gemini to assist in the trouble shooting so some of analytical statements below may be off).

My Environment:

**Graylog Version:** V6.2.3

**OS Version:**
PRETTY_NAME="Ubuntu 24.04.2 LTS"
NAME="Ubuntu"
VERSION_ID="24.04"
VERSION="24.04.2 LTS (Noble Numbat)"
VERSION_CODENAME=noble
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=noble
LOGO=ubuntu-logo

**Java Version:**
openjdk version "21.0.3" 2024-04-16 LTS
OpenJDK Runtime Environment Temurin-21.0.3+9 (build 21.0.3+9-LTS)
OpenJDK 64-Bit Server VM Temurin-21.0.3+9 (build 21.0.3+9-LTS, mixed mode, sharing)

**Mongod Version**
db version v6.0.24
Build Info: {
"version": "6.0.24",
"gitVersion": "1b052b94a23863fd12be97aaa4e4b1d96456e5cc",
"openSSLVersion": "OpenSSL 3.0.13 30 Jan 2024",
"modules": [],
"allocator": "tcmalloc",
"environment": {
"distmod": "ubuntu2204",
"distarch": "x86_64",
"target_arch": "x86_64"
}
}

**The Problem:**
My graylog-datanode service starts, but the main Java process exits immediately with status=1/FAILURE, as seen in the systemd journal.

The primary symptom of this datanode crash loop is a constant "flapping" of the cluster leader status, with "NO_LEADER" alerts appearing and resolving every few seconds in the System Events log.

**The Root Cause:**
The ps aux command shows the OpenSearch Java process is being launched with the -Djava.security.manager=allow flag, which is causing the immediate crash in my environment.

**Troubleshooting Steps Taken (Key Findings):**

1. I confirmed the problematic -Djava.security.manager=allow flag is present in the running process via ps aux.

2. I search for this flag in the user-configurable files. I checked:

* The jvm.options file in the datanode's config directory (both default and override paths).
* All EnvironmentFiles listed in the graylog-datanode.service unit file.
* For any systemd drop-in override files (the override directory does not exist).

3. The process fails early so it creates no log files. The OpenSearch log directory is empty.

4. The only record of the failure is the generic exit code in the systemd journal.

This evidence suggest the setting is being injected by the graylog-datanode packaging or its startup scripts, not by user configuration. Any help in finding a workaround or a patched version would be greatly appreciated.