Azure / Azure/fluentd-plugin-mdsd

ANSI control character in captured strings break MDSD ingestion

Open
#79 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
28
Forks
26
PR merge metrics
No merged PRs in 30d

Description

ANSI control characters that may be used to colorize a TTY, when captured by fluentd and forwarded to mdsd cause the ingestion to fail with an `ACK_DECODE_ERROR`.

Such strings are for instance seen when containers workloads such as [bitnami/openresty docker images](https://hub.docker.com/r/bitnami/openresty) start up and log strings such as the following to stdout/stderr:

`\u001b[38;5;6mopenresty \u001b[38;5;5m21:25:41.40 \u001b[0m\u001b[38;5;2mINFO \u001b[0m ==> Initializing OpenResty`

Note that the above representation is an encoded string that escapes the ANSI control characters.

In order to avoid ingestion to fail, custom filters such as the following have to be added to sanitize the string.

```xml

@type record_transformer
enable_ruby

log ${ record["log"].gsub(/[\u001b\u009b][\[()#;?]*(?:[0-9]{1,4}(?:;[0-9]{0,4})*)?[0-9A-ORZcf-nqry=><]/, '') }

```

As a user of the mdsd plugin I expect the plugin to sanitize or escape any invalid characters out from an `FT_STRING` before it is passed to mdsd based on what mdsd deems valid, without requiring custom filters to be written.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.