fluent / fluent/fluent-operator

bug: fluent-bit process once again hangs sometimes after being restarted

Open
#1,407 3 comments 0 reactions 1 assignee Claimed by @wenchajun View on GitHub
bug
Dominant language
Go
Stars
682
Forks
328
Avg merge
2d 23h
Merged PRs (30d)
13

Description

### Describe the issue

Some time ago the fluentbit-watcher has been reworked to utilise the hot-reload feature
https://github.com/fluent/fluent-operator/commit/90d364bfc1d42187eaddc600b1a6077fa1cfa4f5

This also meant removal of the SIGKILL call when the process is hanging. And so the issue that I initially reported in #510 has been reintroduced.

This is something that ideally would be fixed in fluent-bit itself (and I will report it there as well once I investigate this problem more in-depth and can reproduce it consistently...), but in the meantime I think it would be great to have handling for these situations reintroduced in fluent-operator.

### To Reproduce

No clear steps to reproduce. Seems to happen when fluent-bit is restarted many times in a row, but not always

### Expected behavior

Fluent-bit is restarted and works

### Your Environment

```markdown
- Fluent Operator version:
- Container Runtime:
- Operating system:
- Kernel version:
```

### How did you install fluent operator?

_No response_

### Additional context

Keeping this as somewhat of a remainder go get back to this after 18.11 or so

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.