fluent / fluent/fluentd

in_tail: read_bytes_limit_per_second doesn't work over file rotations

Open
#3,567 1 comment 1 reaction 0 assignees View on GitHub
bug
Dominant language
Ruby
Stars
13.6k
Forks
1.4k
Avg merge
1d 3h
Merged PRs (30d)
20

Description

### Describe the bug

If a logfile is filled out quickly enough, it can be rotated several times per second. The throughput counter however lives in `IOHandler`, which is re-created on file rotation. As a consequence, `read_bytes_limit_per_second` is not respected when the log source is spammy enough.

### To Reproduce

I deployed the following pod in an K8S cluster:
```
apiVersion: v1
kind: Pod
metadata:
name: logflooder
namespace: default
spec:
containers:
- image: ubuntu:bionic
command: ["bash"]
args: ["-c", "while true; do cat /etc/passwd; done"]
imagePullPolicy: IfNotPresent
name: fluentd
resources:
limits:
cpu: "5"
memory: 400Mi
requests:
cpu: "5"
memory: 400Mi
```

The log throughput seemed to effectively constrained by the CPU limit and not the value of `read_bytes_limit_per_second`. The `detected rotation of` message appears several times per second.

### Expected behavior

I'd expect the total log throughput to be bound by `read_bytes_limit_per_second` regardless of file rotations.

### Your Environment

```markdown
- Fluentd version: 1.14.2
- Operating system: Amazon Linux 2
- Kernel version: 4.14.252-195.483.amzn2.x86_64
```

### Your Configuration

```apache
...

@type tail
@id in_tail_container_logs
path "/var/log/containers/*.log"
pos_file "/var/log/fluentd-containers.log.pos"
read_bytes_limit_per_second 100k
tag "kubernetes.*"
exclude_path ["/var/log/containers/fluentd-*"]
read_from_head true

@type "json"
time_format "%Y-%m-%dT%H:%M:%S.%NZ"
unmatched_lines
time_type string


...
```
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.