containerd / containerd/nerdctl

The unexpected exit of `jsonfile.Encode` is blocking the container's stdout and stderr.

Open
#4,591 2 comments 0 reactions 0 assignees View on GitHub
kind/unconfirmed-bug-claim
Dominant language
Go
Stars
10.4k
Forks
826
Avg merge
1d 23h
Merged PRs (30d)
44

Description

### Description

Any file write error will cause `jsonfile.Encode` to exit, stopping the copying of container logs. When buffer is full, container log writes will then block and hang.

Due to a full file system in our environment, a container hung on write logs. Despite our prompt cleanup, the issue persisted and was resolved by restarting the container finally.

relate https://github.com/containerd/containerd/issues/12462

Container use `json-file` log driver have this issue.

https://github.com/containerd/nerdctl/blob/3c376a817e6b825df192a4d3fdb65404e1276f81/pkg/logging/jsonfile/jsonfile.go#L62-L65

### Steps to reproduce the issue

1. `nerdctl run -d --name logging-test docker.io/library/busybox:latest ping 127.0.0.1 -i 0.1 -c 3000`
2. `chattr +i $(nerdctl inspect logging-test -f {{.LogPath}}); sleep 1; chattr -i $(nerdctl inspect logging-test -f {{.LogPath}})`
3. The ping commands hangs forever.

### Describe the results you received and expected

Received: container hang
Expected: container not hang

### What version of nerdctl are you using?

```
Client:
Version: 1.7.7
OS/Arch: linux/amd64
Git commit: 23d838b457213765095992ddd2c1b26a0c402632.m
buildctl:
Version:

Server:
containerd:
Version: 1.6.32
GitCommit: 8b3b7ca2e5ce38e8f31a34f35b2b68ceb8470d89
runc:
Version: 1.1.12
GitCommit: v1.1.12-0-g51d5e94
```

### Are you using a variant of nerdctl? (e.g., Rancher Desktop)

None

### Host information

```
Client:
Namespace: default
Debug Mode: false

Server:
Server Version: 1.6.32
Storage Driver: overlayfs
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Log: fluentd journald json-file syslog
Storage: native overlayfs
Security Options:
seccomp
Profile: builtin
Kernel Version: 4.18.0-513.24.1.el8_9.v1.x86_64
Operating System: Rocky Linux 8.9 (Green Obsidian)
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 1.718GiB
Name: osbuild
ID: ef318042-eff5-4f1c-9609-b726bd1e0a19

WARNING: IPv4 forwarding is disabled
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.