fluent / fluent/fluentd-docs-gitbook

Unclear description of logic that happens on an error in the "parser filter"

Open
#442 2 comments 2 reactions 0 assignees View on GitHub
bug
Dominant language
CSS
Stars
46
Forks
136
Avg merge
4h 47m
Merged PRs (30d)
16

Description

### Describe the bug

`emit_invalid_record_to_error` in the parser filter is a setting which allows users to decide on the outcome that happens when a record is unable to parsed for one of three reasons.
If set to true it says that the record will be given the `@ERROR` label, and "If you want to ignore these errors, set false".

However, I was seeing different behaviour, and looking at the relevant piece of source code https://github.com/fluent/fluentd/blob/c48ca04a593f2f4628c545d74c90f6caba513bee/lib/fluent/plugin/filter_parser.rb#L72-L95
it became clear that the documentation isn't quite clear.

Notably, the source code clearly has an extra condition that if `reserve_data` is true then it will not discard these errors (as I would of expected), but instead it allows them to pass but with an unparsed record. I fully understand why this behaviour exists, but I think the documentation around this could be included.

### Link to the problematic documentation

https://docs.fluentd.org/filter/parser#emit_invalid_record_to_error
https://docs.fluentd.org/filter/parser#reserve_data

### Expected explanation

I would expect the documentation for `emit_invalid_record_to_error` to clearly say that if `reserve_data` is set to true, then the record will not be discarded but will instead be passed through without any parsing happening.

### Additional context

An example configuration which was not clear from the documentation is:
```

@type parser

@type json
json_parser json

key_name log
reserve_data true
emit_invalid_record_to_error false

```

I am happy to open a PR myself, but given this is my first open-source contribution I figured I would put an issue in.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.