aws / aws/containers-roadmap

[ECS Fargate] [request]: Partial_message field not included in platform version 1.4.0

Open
#1,550 8 comments 29 reactions 0 assignees View on GitHub
ECS Fargate Proposed
Dominant language
Shell
Stars
5.4k
Forks
334
PR merge metrics
No merged PRs in 30d

Description

### Community Note

* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue or have submitted a pull request, please leave a comment

**Tell us about your request**

In all container runtimes that I am aware of, stdout/stderr messages from containers are split after a certain limit. In Docker, CRI, and containerd, IIRC the limit is always 16 KB. This means that if you emit messages larger than 16KB from your container, they will be split into multiple messages.

You can then use a tool like Fluentd (Fluent Bit support is something I am working on: https://github.com/aws/aws-for-fluent-bit/issues/25), to concatenate those records back into one. However, this relies on the runtime setting some flag to tell you that the series of messages you are receiving was actually originally one message.

The Fluentd Plugin Concat can join these back together: https://github.com/fluent-plugins-nursery/fluent-plugin-concat

To understand how this works, let's look at the example from the Fluentd plugin unit test: https://github.com/fluent-plugins-nursery/fluent-plugin-concat/blob/master/test/plugin/test_filter_concat.rb#L551

```
sub_test_case "partial_key" do
test "filter with docker style events" do
config = <<-CONFIG
key message
partial_key partial_message
partial_value true
CONFIG
messages = [
{ "container_id" => "1", "message" => "start", "partial_message" => "true" },
{ "container_id" => "1", "message" => " message 1", "partial_message" => "true" },
{ "container_id" => "1", "message" => " message 2", "partial_message" => "true" },
{ "container_id" => "1", "message" => "end", "partial_message" => "false" },
{ "container_id" => "1", "message" => "start", "partial_message" => "true" },
{ "container_id" => "1", "message" => " message 3", "partial_message" => "true" },
{ "container_id" => "1", "message" => " message 4", "partial_message" => "true" },
{ "container_id" => "1", "message" => "end", "partial_message" => "false" },
]
filtered = filter(config, messages, wait: 3)
expected = [
{ "container_id" => "1", "message" => "start\n message 1\n message 2\nend" },
{ "container_id" => "1", "message" => "start\n message 3\n message 4\nend" },
]
assert_equal(expected, filtered)
end
```

**Which service(s) is this request for?**
ECS Fargate

**Are you currently working around this issue?**

I am not aware of any way to work around this.

Contributor guide

Open the contributing guide

Research direction

Start by investigating ECS Fargate platform version 1.4.0 and how container stdout/stderr messages are emitted when they exceed the runtime limit. Compare the behavior with the referenced Fluentd concat test and Docker-style partial_message records. Done means split messages include the metadata needed to identify and concatenate the original message.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, docker
Domain
cloud
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.