bluewave-labs / bluewave-labs/Checkmate

MongoDB fails to start on Linux kernel 6.19+ / 7.0.x (SERVER-121912) - app returns 502, DB connection refused

Open Beginner friendly
#3,842 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
10.8k
Forks
1.2k
Avg merge
1d 4h
Merged PRs (30d)
74

Description

After upgrading my host to a newer Linux kernel (Proxmox VE 9, kernel `7.0.14-12-pve` - Proxmox 8 goes out of support on 31st August 2026 and Proxmox 9.2, which is the current version, ships by default with trixie kernel 7.0.14), Checkmate started returning 502. The Checkmate server was up but could not reach MongoDB, logging repeated `connect ECONNREFUSED 127.0.0.1:27017`. The root cause is not Checkmate itself but a known MongoDB issue: MongoDB 8.0+ refuses to start on Linux kernel 6.19 and newer due to a TCMalloc/rseq incompatibility, exiting at startup with:

```
MongoDB cannot start: Linux kernel versions 6.19 and newer has a known incompatibility with this version of MongoDB. See https://jira.mongodb.org/browse/SERVER-121912 for more information.
```

## Environment

Checkmate installed via the Proxmox community-scripts LXC (Debian container)
MongoDB 8.0.28, then updated to 8.0.29 - both are affected
Host kernel: `7.0.14-12-pve`

## What I tried that did NOT work

Updating MongoDB within the 8.0 series (8.0.28 to 8.0.29): still gated. The startup check in these builds still blocks 6.19+ and does not yet recognise fixed kernels.
Setting `GLIBC_TUNABLES=glibc.pthread.rseq=0`: no effect.

## What DID work

A systemd drop-in that sets the rseq tunable to 1 for the mongod service:

```
mkdir -p /etc/systemd/system/mongod.service.d
cat </etc/systemd/system/mongod.service.d/rseq.conf
[Service]
Environment=GLIBC_TUNABLES=glibc.pthread.rseq=1
EOF
systemctl daemon-reload
systemctl restart mongod
```

After this, mongod starts normally and Checkmate reconnects. I confirmed it survives a full host reboot.

For Docker Compose deployments, the equivalent is adding to the mongo service:

```
environment:
GLIBC_TUNABLES: "glibc.pthread.rseq=1"
```

## Suggestion

It may be worth documenting this in the README/troubleshooting, since anyone on a current kernel (Debian 13, Ubuntu 26.04, Proxmox VE 9, Fedora 43+) will hit it. This is a temporary workaround; the real fix won't land until MongoDB ships a patched TCMalloc. But this has been ongoing since May, meanwhile it's now middle of August and no progress on that.

## Sources

MongoDB Jira SERVER-121912 (TCMalloc rseq bug): https://jira.mongodb.org/browse/SERVER-121912
MongoDB 8.0 release notes (affected kernel range 6.19 through 7.0.13): https://www.mongodb.com/docs/manual/release-notes/8.0/

Contributor guide

Open the contributing guide

Research direction

Start with the README and existing troubleshooting or deployment guidance, then check how MongoDB is described for systemd and Docker Compose installations. Document the Linux kernel 6.19+ startup incompatibility, the GLIBC_TUNABLES workaround, and the affected deployment examples; done means users can restore MongoDB and Checkmate connectivity from the documented steps.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker-compose, mongodb
Domain
documentation, infrastructure
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.