fluent / fluent/fluentd-docs-gitbook
Unclear description of logic that happens on an error in the "parser filter"
- Dominant language
- CSS
- Stars
- 46
- Forks
- 136
- Avg merge
- 4h 47m
- Merged PRs (30d)
- 16
Description
### Describe the bug
`emit_invalid_record_to_error` in the parser filter is a setting which allows users to decide on the outcome that happens when a record is unable to parsed for one of three reasons.
If set to true it says that the record will be given the `@ERROR` label, and "If you want to ignore these errors, set false".
However, I was seeing different behaviour, and looking at the relevant piece of source code https://github.com/fluent/fluentd/blob/c48ca04a593f2f4628c545d74c90f6caba513bee/lib/fluent/plugin/filter_parser.rb#L72-L95
it became clear that the documentation isn't quite clear.
Notably, the source code clearly has an extra condition that if `reserve_data` is true then it will not discard these errors (as I would of expected), but instead it allows them to pass but with an unparsed record. I fully understand why this behaviour exists, but I think the documentation around this could be included.
### Link to the problematic documentation
https://docs.fluentd.org/filter/parser#emit_invalid_record_to_error
https://docs.fluentd.org/filter/parser#reserve_data
### Expected explanation
I would expect the documentation for `emit_invalid_record_to_error` to clearly say that if `reserve_data` is set to true, then the record will not be discarded but will instead be passed through without any parsing happening.
### Additional context
An example configuration which was not clear from the documentation is:
```
@type parser
@type json
json_parser json
key_name log
reserve_data true
emit_invalid_record_to_error false
```
I am happy to open a PR myself, but given this is my first open-source contribution I figured I would put an issue in.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.