[ECS/FireLens] [request]: Use Dots in Fluent log tags generated by FireLens
- Dominant language
- Shell
- Stars
- 5.4k
- Forks
- 334
- PR merge metrics
- No merged PRs in 30d
Description
### Community Note
* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue or have submitted a pull request, please leave a comment
**Tell us about your request**
For background, how FireLens works is explained here: https://aws.amazon.com/blogs/containers/under-the-hood-firelens-for-amazon-ecs-tasks/
In the Fluentd and Fluent Bit community, it is standard practice to separate the "parts" of a tag with the dot (".") character.
For example, the Fluent Bit rewrite tag character supports a syntax to select a part of a tag delimited with dots: https://docs.fluentbit.io/manual/v/1.4/pipeline/filters/rewrite-tag
If the tag is `aa.bb.cc`, then `$TAG[1]` would be `bb`.
I knew that dot delimiting was a standard practice before I designed FireLens. Yet, I made that tag for container logs delimited with dashes instead `{container name}-firelens-{task ID}`. Several people have pointed out on twitter that this is weird. I am sorry. I do not know why I did that.
In the future, I hope to add a feature in the CloudWatch Logs output so that the log stream or group can be named based on parts of the tag. Imagine you could put in your config:
```
log_stream_format $TAG[0]-logs-$TAG[2]
log_group_format myservice-$TAG[2]
```
Basically, customize the name of the stream/group based on parts of the tag.
To enable that, we need a new field in FireLens to use dots in the tag. This unfortunately probably needs to be an opt-in feature, just because we'd be worried about breaking people who have set up match patterns on the existing tags with dashes.
This is very low priority feature request. There are other things which are very urgently needed for logging in ECS/EKS/Fluent Bit, this is more of a nice-to-have.
**Which service(s) is this request for?**
ECS
Contributor guide
Research direction
Start with the linked FireLens background article and Fluent Bit rewrite-tag documentation to understand the current tag format and its consumers. The issue describes an opt-in dot-delimited tag format for ECS FireLens while preserving existing dash-delimited matching; implementation files and tests are not identified in the payload.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws
- Domain
- cloud, observability-sre
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100