fluent / fluent/fluent-plugin-grok-parser

Question regarding multi-format + multi-line

Open
#85 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Ruby
Stars
108
Forks
32
PR merge metrics
No merged PRs in 30d

Description

Hi,

I have a question rather than an issue.

Does the fluent-plugin-grok-parser support multi-line and multi-format?

I am tailing multiple log4j2 log files. Each of them may have a different format and include stack traces or other multi-line messages.

Is it possible to use the grok parser in this situation? If not then any guidance on how to accomplish this would be greatly appreciated.

Here is a copy of my config's `` element:

~~~

@type tail
@id eap_app_log_tail_source
path /var/log/applogs/app1.log, /var/log/applogs/app2.log,/var/log/applogs/app3.log
pos_file /var/log/td-agent/eap-apps.log.pos
path_key log_file
tag jboss.node.eap.app.logs

@type multiline_grok
grok_failure_key grokfailure

pattern ^%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:loglevel} %{JAVACLASS:logger-class} %{GREEDYDATA:message}$
multiline_start_regexp /\d{4}-\d{1,2}-\d{1,2}/


pattern ^%{TIMESTAMP_ISO8601:timestamp}%{SPACE}%{LOGLEVEL:loglevel} \[(?[A-Za-z0-9_ \-]+)\] {} - %{GREEDYDATA:message}$
multiline_start_regexp /\d{4}-\d{1,2}-\d{1,2}/


pattern ^%{LOGLEVEL:loglevel}%{SPACE}%{TIMESTAMP_ISO8601:timestamp} \[(?[A-Za-z0-9_ \-]+)\] - %{GREEDYDATA:message}$
multiline_start_regexp /([Aa]lert|ALERT|[Tt]race|TRACE|[Dd]ebug|DEBUG|[Nn]otice|NOTICE|[Ii]nfo|INFO|[Ww]arn?(?:ing)?|WARN?(?:ING)?|[Ee]rr?(?:or)?|ERR?(?:OR)?|[Cc]rit?(?:ical)?|CRIT?(?:ICAL)?|[Ff]atal|FATAL|[Ss]evere|SEVERE|EMERG(?:ENCY)?|[Ee]merg(?:ency)?))(?:\s*)/

~~~

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.