aws-samples / aws-samples/amazon-ecs-firelens-examples
Issue with FARGATE 1.4.0
- Dominant language
- No language data
- Stars
- 335
- Forks
- 99
- PR merge metrics
- No merged PRs in 30d
Description
Trying to use awsfirelens and send to multiple destinations.
Created fluentbit config and uploaded to S3:
```
[SERVICE]
Flush 1
Grace 30
[OUTPUT]
Name cloudwatch_logs
Match *
log_stream_name fluent-bit-cloudwatch
log_group_name fluent-bit-cloudwatch
region eu-central-1
log_format json/emf
auto_create_group true
[OUTPUT]
Name kafka
Match *
Brokers borker1:9092,broker2:9092,broker3:9092
Topics myTopic
```
And TaskDefintion using yml:
``` yaml
TaskDefinition:
Type: 'AWS::ECS::TaskDefinition'
Properties:
ExecutionRoleArn: !GetAtt
- ECSTaskExecutionRole
- Arn
TaskRoleArn: !GetAtt
- ECSTaskExecutionRole
- Arn
ContainerDefinitions:
- Name: 'log_router'
Image: '906394416424.dkr.ecr.eu-central-1.amazonaws.com/aws-for-fluent-bit:stable'
Essential: true
FirelensConfiguration:
Type: "fluentbit"
Options:
config-file-type: "s3"
config-file-value: "arn:aws:s3:::XXXXXX/fluentbit-service.conf"
LogConfiguration:
LogDriver: awslogs
Options:
awslogs-group: 'log_router'
awslogs-region: !Ref AWS::Region
awslogs-stream-prefix: 'firelens'
- Name: 'logger'
Image: !Sub '${AWS::AccountId}.dkr.ecr.eu-central-1.amazonaws.com/random-logger:latest'
Essential: true
PortMappings:
- HostPort: 80
Protocol: tcp
ContainerPort: 80
LogConfiguration:
LogDriver: awsfirelens
RequiresCompatibilities:
- FARGATE
NetworkMode: awsvpc
Cpu: '256'
Memory: '512'
Family: 'task-family'
```
In Cloudformaion during creation I see all the time message:
```
Resource handler returned message: "One or more of the requested capabilities are not supported.
(Service: AmazonECS; Status Code: 400; Error Code: PlatformTaskDefinitionIncompatibilityException;
Request ID: 3926503c-e2d8-4d02-b385-7619f4a7a5c3; Proxy: null)"
(RequestToken: 32f71748-d116-65e1-185e-467067390ded, HandlerErrorCode: GeneralServiceException)
```
If I set PlatformVersion: 1.3.0 for Service - all works just fine.
Seems like I can't use LogConfiguartion with just LogDriver: awsfirelens and no options.
Tried to add also options for 'cloudwatch' was thinking that it will be added to 2 existing [OUTPUTS] in config file.
But see exactly same issue.
awsfirelens also works with "file" config-file-type.
But clearly there is issue when config-file-type==s3 and LogConfigration has just LogDriver: awsfirelens but no options.
Contributor guide
Research direction
Start with the CloudFormation TaskDefinition and the Fluent Bit configuration shown in the issue. Reproduce the failure using Fargate platform versions 1.3.0 and 1.4.0, comparing an S3 config with awsfirelens configured without options. Done means establishing whether this combination is supported and documenting or correcting the required configuration.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws
- Domain
- cloud, devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100