time_key default needlessly overridden in json, regexp, and ltsv parsers
- Dominant language
- Ruby
- Stars
- 13.6k
- Forks
- 1.4k
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 20
Description
**Describe the bug**
The global default for `time_key` is `nil`, indicating that the event time should not be derived from the message at all, but the json, regexp, and ltsv parsers override this to `"time"`. Furthermore, the fluentd config syntax does not permit `time_key` to be set back to `nil`. as specifying `nil`, `null`, or `""` result in the literal string representations `"nil"`, `"null"`, and `""` being used. Specifying `"#{use_nil}"` results in an error that a value is required.
While it appears that `time_key` was required to be defined in the [original implementations](https://github.com/fluent/fluentd/commit/2f261882d4598af769a81a8e3056ecfb8734b7bf#diff-73e8d1657dbede482feb06b87014a3656632e7e507cd48fd7be2440aa848309e) of these parsers, it has not been required for some time now, and the result is an edge case where a message contains a field called `time` that is not parseable as time causes an exception and the message to be dropped.
It is possible to work around this issue by setting `time_key` to a value that you do not expect to occur as a key in your messages, but this is a kludge at best.
It's also worth noting here that the docs _do_ note that `time_key` is overridden by these parsers: https://docs.fluentd.org/configuration/parse-section#parse-parameters
**To Reproduce**
1. Define any input with a json parser.
2. Submit a message such as `{"time": {"begin": 1611709938020}}`
**Expected behavior**
The message to be decoded as a dict with a key named "time" and its arbitrary value.
**Environment**
- Fluentd or td-agent version: 1.11.5
- Operating system: docker image `fluent/fluentd:v1.11.5-1.0`
**Configuration**
```
# ...
@type json
```
**Error Log**
```
#1611709938020}(Hash)> @ /usr/lib/ruby/gems/2.7.0/gems/fluentd-1.11.5/lib/fluent/plugin/parser.rb:196:in `rescue in parse_time'
```
Contributor guide
Assessment
This issue has not been assessed yet.