fluent / fluent/fluent-bit

Making Parser-types as plugins

Open
#889 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
C
Stars
8.1k
Forks
2k
Avg merge
4d 16h
Merged PRs (30d)
58

Description

**Is your feature request related to a problem? Please describe.**
I have made new parser filter for parsing MariaDB audit-log events (from syslog)
It's actually general of 'csv'-parser (omission of double-quoted).
I would like to change it to new parser, but I see that current
parser support in https://github.com/fluent/fluent-bit/blob/master/src/flb_parser.c
is very much "hard-coded".
Especially adding new configuration for parser is not very clean when
parsers have common configuration function.
And I suppose that there will be other new parsers in future too.

**Describe the solution you'd like**
I would like to have "plugin" like structure for parsers,
Where we would have **flb_parser_plugin**-struct for parsers like there is for filters and other plugins.
So parses would have their own ```cb_init```, ```cb_parse```, ```cb_exit``` methods and
own context for configuration.

I have looked current code and it's seems to quite simple change (I haven't seen corner cases).

I'm ready to implement this... if there seems to support for idea.

**Describe alternatives you've considered**
Other ways:
- Just to add new filter normal old way... (more hard-coding)
- Have this as filter as my current implementation:
https://github.com/bluebike/fluent-bit/tree/filter_csv_module

**Additional context**
MariaDB audit log format : https://mariadb.com/kb/en/library/mariadb-audit-plugin-log-format/

Problem with audit log format is that comma (,) separated, but contains quoted ('...') strings, which can contain backslash escaped chars (also "," and "'"), so regex parsing doesn't work very well with it.

We are currently collecting audit events using *in_syslog*.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.