fluent / fluent/fluentd

Incorrect error count metrics when using buffer in output plugin

Open
#3,215 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Ruby
Stars
13.6k
Forks
1.4k
Avg merge
1d 3h
Merged PRs (30d)
20

Description

Check [CONTRIBUTING guideline](https://github.com/fluent/fluentd/blob/master/CONTRIBUTING.md) first and here is the list to help us investigate the problem.

**Describe the bug**
When using a buffer, the num_errors reports all errors occurred, even if some of them are resolved by a retry.

For example, when using GCS output plugin, every error will be reported - even though this is solved after a couple of retries.

Looking at the [code](https://github.com/fluent/fluentd/blob/36d017deb6b37a55833691edbab8d5f31353ac36/lib/fluent/plugin/output.rb#L817), I noticed there is already a TODO to fix that so I guess this is a known issue. I'll be happy to help fixing it - just need some help to discuss the solution :)

**To Reproduce**
1. Create output plugin that produce random error and a buffer
2. Use it and emit events
3. Notice the `num_errors` is increased, even thought the retry eventually solves the error and the event are sent.

**Expected behavior**
Move errors to a different counter and report an error only when reaching the retry limit.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.