[ECS] [request]: Support Mem_Buf_Limit in FireLens
- Dominant language
- Shell
- Stars
- 5.4k
- Forks
- 334
- PR merge metrics
- No merged PRs in 30d
Description
### Community Note
* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue or have submitted a pull request, please leave a comment
**Tell us about your request**
With FireLens, the[ input definitions for Fluent Bit are generated by ECS](https://aws.amazon.com/blogs/containers/under-the-hood-firelens-for-amazon-ecs-tasks/). This prevents customers from setting any custom options on the input configuration. `Mem_Buf_Limit` is an input configuration option which sets the total memory available for buffering logs.
This field should probably be optionally configurable by customers. We should determine if there are any other input options as well which might need to be configured.
UPDATE: We will likely work with Fluent Bit upstream to contribute this instead: https://github.com/fluent/fluent-bit/discussions/5711
**Which service(s) is this request for?**
ECS EC2, and ECS Fargate
**Are you currently working around this issue?**
There's a way to "hack" in input configuration, which is not ideal but I should possibly publish a tutorial on if it is desired...
**EDIT:** Here is the detailed tutorial: https://aws.amazon.com/blogs/containers/how-to-set-fluentd-and-fluent-bit-input-parameters-in-firelens/
Contributor guide
Research direction
Read the FireLens input-generation description and the linked Fluent Bit discussion first; the issue does not identify a file or test in this roadmap repository. Confirm the upstream approach and whether ECS EC2 and Fargate support is delivered; done is support for configurable Mem_Buf_Limit and any agreed input options, rather than a roadmap-only change.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws
- Domain
- cloud
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 15/100