in_tail: possibly collects duplicated logs in `rotate_wait`
- Dominant language
- Ruby
- Stars
- 13.6k
- Forks
- 1.4k
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 20
Description
### Describe the bug
I think the major log duplication problem about `follow_inodes` is fixed in #4237.
However, there is still a possibility of duplication (https://github.com/fluent/fluentd/pull/4237#issuecomment-1633358632).
`in_tail` possibly collects duplicated logs in `rotate_wait`.
Mulitple `TailWatcher`s can exist in `rotate_wait`, so they possibly collect the same target in that interval.
We need the mechanism not to add duplicated `TailWathcer` while detaching it.
I also wrote the code comment as follows in #4237:
https://github.com/fluent/fluentd/blob/d5685ada81ac89a35a79965f1e94bbe5952a5d3a/lib/fluent/plugin/in_tail.rb#L507-L515
This is not something like that all logs in one file are entirely duplicated.
(I believe such problems were resolved in #4237)
---
Detail explanation (from https://github.com/fluent/fluentd/pull/4237#issuecomment-1633358632)
The scenario: rotate -> update_watcher -> refresh_watcher -> `rotate_wait` elapsed
* Initial state
```
path-1: inode-b, TailWatcher-1
path-2: inode-a, TailWathcer-2
```
* Rotate happens
```
path-1: inode-c, TailWatcher-1-new
path-2: inode-b, TailWathcer-2-new, TailWatcher-1-detaching
path-3: inode-a, TailWatcher-2-detaching
```
* refresh_watcher
```
path-1: inode-c, TailWatcher-1-new
path-2: inode-b, TailWathcer-2-new, TailWatcher-1-detaching
path-3: inode-a, TailWatcher-2-detaching, TailWatcher-3
```
* `rotate_wait` elapsed
```
path-1: inode-c, TailWatcher-1-new
path-2: inode-b, TailWathcer-2-new
path-3: inode-a, TailWatcher-3
```
As above, `rotate_wait` causes the situation that multiple TailWatchers exist for the same target temporarily.
This causes log duplication.
I have confirmed this on v1.16.1.
### To Reproduce
From https://github.com/fluent/fluentd/pull/4237#issuecomment-1633358632.
Use the config below.
* Start to tail the following file
```
test.log
```
* Rotate
```
test.log
test.log.1
```
* Append a log to `test.log.1`.
* When `refresh_interval ` NOT elapsed and `rotate_wait` NOT elapsed
* No duplication
* When `refresh_interval ` elapsed and `rotate_wait` NOT elapsed
* DUPLICATION
* When `refresh_interval ` elapsed and `rotate_wait` elapsed
* No duplication
### Expected behavior
Log duplication does not occur.
### Your Environment
```markdown
- Fluentd version: 1.16.1, 1.16.2
- Operating system: Ubuntu 20.04.6 LTS
- Kernel version: 5.15.0-71-generic
```
### Your Configuration
```apache
@type tail
tag test
path /path/to/test.log*
pos_file /test/fluentd/pos/pos
follow_inodes true
refresh_interval 5s
enable_stat_watcher false # To ensure that TailWathcer recognizes rotation
rotate_wait 30s
@type none
@type stdout
```
### Your Error Log
```shell
No error.
```
### Additional context
We can set `rotate_wait 0` for a workaround.
Contributor guide
Assessment
This issue has not been assessed yet.