aws-samples / aws-samples/amazon-ecs-firelens-examples

Issue with FARGATE 1.4.0

Open
#59 4 comments 2 reactions 0 assignees View on GitHub
Dominant language
No language data
Stars
335
Forks
99
PR merge metrics
No merged PRs in 30d

Description

Trying to use awsfirelens and send to multiple destinations.
Created fluentbit config and uploaded to S3:

```
[SERVICE]
Flush 1
Grace 30

[OUTPUT]
Name cloudwatch_logs
Match *
log_stream_name fluent-bit-cloudwatch
log_group_name fluent-bit-cloudwatch
region eu-central-1
log_format json/emf
auto_create_group true

[OUTPUT]
Name kafka
Match *
Brokers borker1:9092,broker2:9092,broker3:9092
Topics myTopic
```

And TaskDefintion using yml:

``` yaml
TaskDefinition:
Type: 'AWS::ECS::TaskDefinition'
Properties:
ExecutionRoleArn: !GetAtt
- ECSTaskExecutionRole
- Arn
TaskRoleArn: !GetAtt
- ECSTaskExecutionRole
- Arn
ContainerDefinitions:
- Name: 'log_router'
Image: '906394416424.dkr.ecr.eu-central-1.amazonaws.com/aws-for-fluent-bit:stable'
Essential: true
FirelensConfiguration:
Type: "fluentbit"
Options:
config-file-type: "s3"
config-file-value: "arn:aws:s3:::XXXXXX/fluentbit-service.conf"
LogConfiguration:
LogDriver: awslogs
Options:
awslogs-group: 'log_router'
awslogs-region: !Ref AWS::Region
awslogs-stream-prefix: 'firelens'
- Name: 'logger'
Image: !Sub '${AWS::AccountId}.dkr.ecr.eu-central-1.amazonaws.com/random-logger:latest'
Essential: true
PortMappings:
- HostPort: 80
Protocol: tcp
ContainerPort: 80
LogConfiguration:
LogDriver: awsfirelens
RequiresCompatibilities:
- FARGATE
NetworkMode: awsvpc
Cpu: '256'
Memory: '512'
Family: 'task-family'
```

In Cloudformaion during creation I see all the time message:

```
Resource handler returned message: "One or more of the requested capabilities are not supported.
(Service: AmazonECS; Status Code: 400; Error Code: PlatformTaskDefinitionIncompatibilityException;
Request ID: 3926503c-e2d8-4d02-b385-7619f4a7a5c3; Proxy: null)"
(RequestToken: 32f71748-d116-65e1-185e-467067390ded, HandlerErrorCode: GeneralServiceException)

```

If I set PlatformVersion: 1.3.0 for Service - all works just fine.
Seems like I can't use LogConfiguartion with just LogDriver: awsfirelens and no options.
Tried to add also options for 'cloudwatch' was thinking that it will be added to 2 existing [OUTPUTS] in config file.
But see exactly same issue.
awsfirelens also works with "file" config-file-type.
But clearly there is issue when config-file-type==s3 and LogConfigration has just LogDriver: awsfirelens but no options.

Contributor guide

Open the contributing guide

Research direction

Start with the CloudFormation TaskDefinition and the Fluent Bit configuration shown in the issue. Reproduce the failure using Fargate platform versions 1.3.0 and 1.4.0, comparing an S3 config with awsfirelens configured without options. Done means establishing whether this combination is supported and documenting or correcting the required configuration.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws
Domain
cloud, devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.