containerd / containerd/containerd

Continuous memory growth in containerd v2.1.4

Open
#12,738 20 comments 2 reactions 0 assignees View on GitHub
area/runtime kind/bug
Dominant language
Go
Stars
21.3k
Forks
4.1k
Avg merge
2d 4h
Merged PRs (30d)
92

Description

### Description

We are seeing continuous memory growth on `containerd` version `github.com/containerd/containerd/v2 2.1.4` in our AWS EKS v1.32 cluster.

The containerd process memory grows constantly and reaches maximum capacity and causes the node to become non-responsive and also causes OS to kill other things abruptly due to memory pressure causing node to go into `NotReady` status.

### Steps to reproduce the issue

We simply keep it running for a few days and memory grows.

(Note: We have duplicate metrics. So two same value lines are shown. Ignore one)

Image

### Describe the results you received and expected

Memory should not grow unbounded like this.

### What version of containerd are you using?

containerd github.com/containerd/containerd/v2 2.1.4 75cb2b7193e4e490e9fbdc236c0e811ccaba3376

### Any other relevant information

We also use these nodes for running periodic Argo jobs (short-lived jobs run periodically every 2mins, etc.). I am not sure if this has any relation.

### Show configuration if it is related to CRI plugin.

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.