[AWS Batch] awsfirelens Log Driver Support in AWS Batch with Fargate
- Dominant language
- Shell
- Stars
- 5.4k
- Forks
- 334
- PR merge metrics
- No merged PRs in 30d
Description
I'm trying to configure awsfirelens as the log driver for my AWS Batch jobs running on Fargate, similar to the approach described in this [article](https://docs.datadoghq.com/integrations/ecs_fargate/?tab=webui#fluent-bit-and-firelens). According to the documentation, it appears that awsfirelens is not currently supported for AWS Batch.
However, when creating a JobDefinition (type: Container, orchestration type: Fargate) via the AWS Console, I am able to configure both the awsfirelens log driver and the FirelensConfiguration block without issue. Please see the attached screenshots for reference:
When attempting to replicate this configuration via the API (using Terraform), I receive the following error:
`Error: updating Batch Job Definition (): operation error Batch: RegisterJobDefinition, https response error StatusCode: 400, RequestID: 04e3cbdb-ba02-456a-9e65-caf583224da6, ClientException: Error executing request, Exception : When awsfirelens log driver is specified in log configuration, a firelens configuration object must be configured in one of the containers, RequestId: 04e3cbdb-ba02-456a-9e65-caf583224da6`
This suggests the awsfirelens log driver shouldn't be available, yet it seems to work when set up via the Console. I'm wondering:
Is this an unsupported configuration that happens to work through the Console?
Is this a bug or an upcoming feature that’s being progressively rolled out?
Are there any plans to officially support awsfirelens in AWS Batch on Fargate?
I currently use awsfirelens in my ECS services and would greatly benefit from having consistent logging across services, including Batch.
Any clarification or updates would be greatly appreciated.
Thank you!
Contributor guide
Research direction
Start by reproducing the configuration through Terraform and the AWS Batch RegisterJobDefinition API, then compare it with the AWS Console setup and the documented AWS Batch/Fargate support. Done means the issue has a clear answer on whether awsfirelens and FirelensConfiguration are supported, a bug, or planned, with any required support-status or documentation update identified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, terraform
- Domain
- cloud
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100