couchspawnkillable: Cannot allocate memory
- Dominant language
- Erlang
- Stars
- 7k
- Forks
- 1.1k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 9
Description
## Description
I use `apache/couchdb:3.3.2` docker image on my server.
Today couchdb process crashed because no free memory on the server.
CouchDB logs contians
> -- Logs begin at Tue 2024-01-09 16:32:03 UTC, end at Tue 2024-02-20 22:09:12 UTC. --
Feb 20 20:49:59 ippbx-0a.example.com docker[999209]: sh: 1: exec: /opt/couchdb/bin/../lib/couch-3.3.2/priv/couchspawnkillable: Cannot allocate memory
Feb 20 20:49:59 ippbx-0a.example.com docker[999209]: sh: 1: exec: sh: 1: exec: /opt/couchdb/bin/../lib/couch-3.3.2/priv/couchspawnkillable: Cannot allocate memory/opt/couchdb/bin/../lib/couch-3.3.2/priv/couchspawnkillable: Cannot allocate memorysh: 1: exec: sh: 1: exec: /opt/couchdb/bin/../lib/couch-3.3.2/priv/couchspawnkillable: Cannot allocate memory
Feb 20 20:49:59 ippbx-0a.example.com docker[999209]: /opt/couchdb/bin/../lib/couch-3.3.2/priv/couchspawnkillable: Cannot allocate memory
Feb 20 20:49:59 ippbx-0a.example.com docker[999209]: ^M
Feb 20 20:49:59 ippbx-0a.example.com docker[999209]: sh: 1: exec: sh: 1: exec: /opt/couchdb/bin/../lib/couch-3.3.2/priv/couchspawnkillable: Cannot allocate memory
Feb 20 20:49:59 ippbx-0a.example.com docker[999209]: sh: 1: exec:
Feb 20 20:49:59 ippbx-0a.example.com docker[999209]: /opt/couchdb/bin/../lib/couch-3.3.2/priv/couchspawnkillable: Cannot allocate memory
Feb 20 20:50:07 ippbx-0a.example.com docker[999209]: out of memory
Feb 20 21:06:23 ippbx-0a.example.com docker[999209]: out of memory
Feb 20 21:10:08 ippbx-0a.example.com docker[999209]: eheap_alloc: Cannot allocate 71302344 bytes of memory (of type "heap").
Feb 20 21:10:08 ippbx-0a.example.com docker[999209]: ^M^M
Feb 20 21:19:07 ippbx-0a.example.com docker[999209]: Crash dump is being written to: erl_crash.dump...
Feb 20 21:19:07 ippbx-0a.example.com systemd[1]: couchdb-docker.service: Main process exited, code=exited, status=137/n/a
Feb 20 21:19:07 ippbx-0a.example.com systemd[1]: couchdb-docker.service: Failed with result 'exit-code'.
Feb 20 21:19:07 ippbx-0a.example.com systemd[1]: couchdb-docker.service: Service RestartSec=100ms expired, scheduling restart.
Feb 20 21:19:07 ippbx-0a.example.com systemd[1]: couchdb-docker.service: Scheduled restart job, restart counter is at 2.
Feb 20 21:19:07 ippbx-0a.example.com systemd[1]: Stopped CouchDB container.
## Steps to Reproduce
Do not know
## Expected Behaviour
All external processes should not lead to all server memory usage.
## Your Environment
* CouchDB version used: 3.3.2
* Browser name and version: do not used
* Operating system and version: CentOS 8 Stream, aarch64
## Additional Context
Here is graph for swap memory usage on the server. Swap usage start grow about one week ago

And the disk cache started shrinking about 8 days ago

A related ticket found at https://issues.apache.org/jira/browse/COUCHDB-371 and looks like issue is not solved and closed with comment
> If this issue remains in CouchDB 2.x, please file a new issue
Contributor guide
Research direction
Start by reproducing the failure with the apache/couchdb:3.3.2 Docker image on CentOS 8 Stream aarch64 and inspect the reported couchspawnkillable, out-of-memory, and Erlang crash-dump logs. No source file or test is named; done would require a reproducible case and evidence that external processes cannot exhaust all server memory.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, erlang
- Domain
- databases, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 18/100