containers / containers/conmon
Max log size not applied when container restarts
- Dominant language
- C
- Stars
- 499
- Forks
- 150
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 20
Description
The max log size option is set to 200MB through `log_size_max = 209715200` in /etc/containers/containers.conf (no other option is uncommented in it), but the container logs exceed 200MB. This seems to happen when a container restarts and thus the conmon process is restarted. In my case, podman restarts the container every few minutes and the bytes written by the previous conmon process do not seem to be taken in count when applying the max log size.
**Versions**
`CentOS 7`
`conmon --version`
`conmon version 2.0.21
commit: efe288263ebb1bab4673d2cfbe1a352f3c7d27c1-dirty`
`runc --version`
`runc version 1.0.0-rc10
commit: 4791ca3d31174576ddd8f4ff562283339196efa9
spec: 1.0.1-dev`
`rpm -qa | grep containers`
`containers-common-1.2.0-3.el7.x86_64`
`podman version`
`Version: 2.1.1
API Version: 2.0.0
Go Version: go1.15.2
Built: Wed Oct 28 07:37:02 2020
OS/Arch: linux/amd64`
**Results obtained**
This container's log file is over 800MB:
`ls -lah /var/lib/containers/storage/overlay-containers/0df62dc3df3784312e28227fa8937e143ad3ce1b87ea112af7b99ac08d23a9e7/userdata/ctr.log`
`-rw-------. 1 root root 839M Jun 1 14:05 /var/lib/containers/storage/overlay-containers/0df62dc3df3784312e28227fa8937e143ad3ce1b87ea112af7b99ac08d23a9e7/userdata/ctr.log`
I can see that the --log-size-max option is applied with the right value by conmon.
` ps -ef | grep conmon | grep /var/lib/containers/storage/overlay-containers/0df62dc3df3784312e28227fa8937e143ad3ce1b87ea112af7b99ac08d23a9e7/userdata/ctr.log
root 23953 1 0 14:04 ? 00:00:00 /usr/bin/conmon --api-version 1 -c 0df62dc3df3784312e28227fa8937e143ad3ce1b87ea112af7b99ac08d23a9e7 -u 0df62dc3df3784312e28227fa8937e143ad3ce1b87ea112af7b99ac08d23a9e7 -r /usr/bin/runc -b /var/lib/containers/storage/overlay-containers/0df62dc3df3784312e28227fa8937e143ad3ce1b87ea112af7b99ac08d23a9e7/userdata -p /var/run/containers/storage/overlay-containers/0df62dc3df3784312e28227fa8937e143ad3ce1b87ea112af7b99ac08d23a9e7/userdata/pidfile -n fluent-sd-pod-fluent-sd --exit-dir /var/run/libpod/exits --socket-dir-path /var/run/libpod/socket -s -l k8s-file:/var/lib/containers/storage/overlay-containers/0df62dc3df3784312e28227fa8937e143ad3ce1b87ea112af7b99ac08d23a9e7/userdata/ctr.log --log-size-max 209715200 --log-level error --runtime-arg --log-format=json --runtime-arg --log --runtime-arg=/var/run/containers/storage/overlay-containers/0df62dc3df3784312e28227fa8937e143ad3ce1b87ea112af7b99ac08d23a9e7/userdata/oci-log --conmon-pidfile /var/run/containers/storage/overlay-containers/0df62dc3df3784312e28227fa8937e143ad3ce1b87ea112af7b99ac08d23a9e7/userdata/conmon.pid --exit-command /usr/bin/podman --exit-command-arg --root --exit-command-arg /var/lib/containers/storage --exit-command-arg --runroot --exit-command-arg /var/run/containers/storage --exit-command-arg --log-level --exit-command-arg error --exit-command-arg --cgroup-manager --exit-command-arg systemd --exit-command-arg --tmpdir --exit-command-arg /var/run/libpod --exit-command-arg --runtime --exit-command-arg runc --exit-command-arg --storage-driver --exit-command-arg overlay --exit-command-arg --storage-opt --exit-command-arg overlay.mountopt=nodev --exit-command-arg --events-backend --exit-command-arg journald --exit-command-arg container --exit-command-arg cleanup --exit-command-arg 0df62dc3df3784312e28227fa8937e143ad3ce1b87ea112af7b99ac08d23a9e7`
Container has been restarted recently:
`date`
`Tue Jun 1 14:36:06 CDT 2021`
`ps -eo pid,lstart,cmd | grep /var/lib/containers/storage/overlay-containers/0df62dc3df3784312e28227fa8937e143ad3ce1b87ea112af7b99ac08d23a9e7/userdata/ctr.log`
`17386 Tue Jun 1 14:32:21 2021 <...>`
**Results expected**
Container log file size not to exceed 200MB.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the restart scenario with /etc/containers/containers.conf, conmon --log-size-max 209715200, and the container's userdata/ctr.log. Compare log handling before and after conmon restarts; done means the log file remains at or below 200MB across container restarts.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c
- Domain
- devtools
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100