Azure / Azure/fluentd-plugin-mdsd
ANSI control character in captured strings break MDSD ingestion
- Dominant language
- C++
- Stars
- 28
- Forks
- 26
- PR merge metrics
- No merged PRs in 30d
Description
ANSI control characters that may be used to colorize a TTY, when captured by fluentd and forwarded to mdsd cause the ingestion to fail with an `ACK_DECODE_ERROR`.
Such strings are for instance seen when containers workloads such as [bitnami/openresty docker images](https://hub.docker.com/r/bitnami/openresty) start up and log strings such as the following to stdout/stderr:
`\u001b[38;5;6mopenresty \u001b[38;5;5m21:25:41.40 \u001b[0m\u001b[38;5;2mINFO \u001b[0m ==> Initializing OpenResty`
Note that the above representation is an encoded string that escapes the ANSI control characters.
In order to avoid ingestion to fail, custom filters such as the following have to be added to sanitize the string.
```xml
@type record_transformer
enable_ruby
log ${ record["log"].gsub(/[\u001b\u009b][\[()#;?]*(?:[0-9]{1,4}(?:;[0-9]{0,4})*)?[0-9A-ORZcf-nqry=><]/, '') }
```
As a user of the mdsd plugin I expect the plugin to sanitize or escape any invalid characters out from an `FT_STRING` before it is passed to mdsd based on what mdsd deems valid, without requiring custom filters to be written.
Contributor guide
Assessment
This issue has not been assessed yet.