influxdata / influxdata/kapacitor
Sessions service log messages produce invalid logfmt?
- Dominant language
- Go
- Stars
- 2.4k
- Forks
- 479
- Avg merge
- 4d 16h
- Merged PRs (30d)
- 4
Description
```
ts=2018-12-22T20:19:26.802Z lvl=info msg="created log session" service=sessions id=d9d9969a-2b87-4661-842c-4070cf625ea2 content-type=application/logfmt tags_0=task=mock_stream_task
ts=2018-12-22T20:19:35.140Z lvl=info msg="deleted log session" service=sessions id=d9d9969a-2b87-4661-842c-4070cf625ea2 content-type=application/logfmt tags_0=task=mock_stream_task
ts=2018-12-22T20:19:59.010Z lvl=info msg="deleted log session" service=sessions id=f70c1aec-de9f-427e-ad0f-99aeb4520ac0 content-type=application/json tags_0=task=mock_stream_task
```
The `tags_0` key has values with `=`: `task=mock_stream_task`
[hutils](https://github.com/brandur/hutils) will parse and truncate `tags_0=task=mock_stream_task` to `tags_0` = `tasks`
```
$ echo 'ts=2018-12-22T20:19:59.010Z lvl=info msg="deleted log session" service=sessions id=f70c1aec-de9f-427e-ad0f-99aeb4520ac0 content-type=application/json tags_0=task=mock_stream_task' | lcut tags_0
task
```
But telegraf tail input with data_format=logfmt fails with error:
```
Dec 21 01:06:33 my_host.local telegraf[15160]: 2018-12-21T01:06:33Z E! [inputs.tail]: Error in plugin: E! Malformed log line in /var/log/kapacitor/kapacitor.log: [ts=2018-12-21T01:06:33.889Z lvl=info msg="deleted log session" service=sessions id=2224d0d7-aedb-4145-abf2-de5c95c818a3 content-type=application/logfmt tags_0=task=mytask], Error: logfmt syntax error at pos 164 on line 1: unexpected '='
```
I'm not able to find anything that specifies how to escape `=` in logfmt. So I'm not sure if it's valid in values. Reporting issue here first under the assumption that it's not valid. If you guys think this should be handled by logfmt consumer I can report in telelgraf repo.
Contributor guide
Research direction
Start by tracing where the sessions service produces the shown logfmt fields, then compare its output with the logfmt parsing behavior described for hutils and Telegraf. Determine whether the producer output or consumer parsing is responsible for the malformed line. Done means the intended session log entries are accepted and parsed without truncation or syntax errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100