NFS: Stale file handle - after log cycling occurs.
- Dominant language
- Go
- Stars
- 4k
- Forks
- 392
- PR merge metrics
- No merged PRs in 30d
Description
I'm running mtail as a separate Docker container, mounting an NFS (EFS) volume to which another application writes out it's log files. I noticed however that at some point the mtail containers went to 100% CPU, closer investigation shows that the mtail logs are rapidly filling up with:
```
I0422 19:46:12.453819 13 filestream.go:116] read /logs/app.log: stale NFS file handle
I0422 19:46:12.453841 13 filestream.go:116] read /logs/app.log: stale NFS file handle
I0422 19:46:12.453872 13 filestream.go:116] read /logs/app.log: stale NFS file handle
...
...
```
That explains the 100% CPU at least, I guess. However, I found out that I can trigger this behaviour by simply redeploying the application that is writing out the logs (app.log will be renamed to app.log.1 on deployment and a fresh app.log is written out).
I'm understanding that mtail has some heuristics to attempt to follow files after rotation, is there a particular reason why this is failing when the files are served from NFS?
Contributor guide
Assessment
This issue has not been assessed yet.