aws / aws/containers-roadmap

[ECS] [request]: Support Mem_Buf_Limit in FireLens

Open
#964 39 comments 43 reactions 0 assignees View on GitHub
ECS FluentBit Proposed
Dominant language
Shell
Stars
5.4k
Forks
334
PR merge metrics
No merged PRs in 30d

Description

### Community Note

* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue or have submitted a pull request, please leave a comment

**Tell us about your request**

With FireLens, the[ input definitions for Fluent Bit are generated by ECS](https://aws.amazon.com/blogs/containers/under-the-hood-firelens-for-amazon-ecs-tasks/). This prevents customers from setting any custom options on the input configuration. `Mem_Buf_Limit` is an input configuration option which sets the total memory available for buffering logs.

This field should probably be optionally configurable by customers. We should determine if there are any other input options as well which might need to be configured.

UPDATE: We will likely work with Fluent Bit upstream to contribute this instead: https://github.com/fluent/fluent-bit/discussions/5711

**Which service(s) is this request for?**
ECS EC2, and ECS Fargate

**Are you currently working around this issue?**

There's a way to "hack" in input configuration, which is not ideal but I should possibly publish a tutorial on if it is desired...

**EDIT:** Here is the detailed tutorial: https://aws.amazon.com/blogs/containers/how-to-set-fluentd-and-fluent-bit-input-parameters-in-firelens/

Contributor guide

Open the contributing guide

Research direction

Read the FireLens input-generation description and the linked Fluent Bit discussion first; the issue does not identify a file or test in this roadmap repository. Confirm the upstream approach and whether ECS EC2 and Fargate support is delivered; done is support for configurable Mem_Buf_Limit and any agreed input options, rather than a roadmap-only change.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws
Domain
cloud
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
15/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.