aws-cloudformation / aws-cloudformation/cloudformation-coverage-roadmap
[AWS::Batch::JobQueue] - [Enhancement] - Replacement Without Interrupting Running Jobs
- Dominant language
- No language data
- Stars
- 1.1k
- Forks
- 62
- PR merge metrics
- No merged PRs in 30d
Description
### Name of the resource
AWS::Batch::JobQueue
### Resource name
_No response_
### Description
### Problem
Updating properties that require replacement on `AWS::Batch::JobQueue` with a fixed `JobQueueName` fails with 409 ConflictException. CloudFormation's create-before-delete pattern tries to create a new queue with the same name before deleting the old one.
**Critical: Customers cannot safely update JobQueues without either deployment failures (409 errors) or interrupting running jobs (when using workarounds).**
Why customers use fixed names:
- External services submit jobs via AWS SDK using the queue name/ARN
- Stable identifiers required for third-party integrations
When customers attempt workarounds (like renaming), CloudFormation deletes the queue while jobs are still running, causing service disruption.
## Current Behavior
1. Update property requiring replacement (e.g., `Tags`)
2. CloudFormation attempts CreateJobQueue with same name
3. Fails with: `ConflictException: Object already exists (Status Code: 409)`
4. Stack rolls back to UPDATE_ROLLBACK_COMPLETE
## Properties Requiring Replacement
Per [CloudFormation docs](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-batch-jobqueue.html):
- `Tags` (most common)
- `JobQueueName`
- `JobQueueType`
Note: [Batch API](https://docs.aws.amazon.com/batch/latest/APIReference/API_TagResource.html) supports updating tags in-place via `TagResource`/`UntagResource`, but CloudFormation resource provider requires replacement.
## Requested Solution
**Primary: Graceful replacement that protects running jobs**
When replacement is required with a fixed name:
1. Disable old queue (prevents new submissions)
2. Wait for running jobs to complete (configurable timeout, default 30 min)
3. Delete old queue
4. Create new queue with same name
5. Enable new queue
If timeout reached: fail safely, leave queue DISABLED, allow customer to investigate.
**Alternative: Make `Tags` updateable in-place**
Use Batch `TagResource`/`UntagResource` APIs to avoid replacement entirely (eliminates most common trigger).
### Other Details
### Current Workarounds
**Documented workarounds:**
- Add `DeletionPolicy: Retain`, remove from stack, manually update tags, re-import
**Community ([GitHub #1229](https://github.com/aws-cloudformation/cloudformation-coverage-roadmap/issues/1229#issuecomment-1407088842)):**
- Rename JobQueue (breaks external integrations)
**All workarounds:**
- Require manual intervention
- Break CloudFormation drift detection
- **Do not guarantee running jobs won't be interrupted**
## Related Issues
- [#1229](https://github.com/aws-cloudformation/cloudformation-coverage-roadmap/issues/1229): JobQueue tag update failures (different error, same root cause)
- [#1655](https://github.com/aws-cloudformation/cloudformation-coverage-roadmap/issues/1655): SchedulingPolicy tags require replacement
- [#1771](https://github.com/aws-cloudformation/cloudformation-coverage-roadmap/issues/1771): Batch resources missing CloudFormation built-in tags
## Suggested Implementation
One possible approach:
- Detect fixed `JobQueueName` during replacement
- Use delete-before-create with job draining:
- `UpdateJobQueue` to DISABLED
- Poll `ListJobs` until all jobs reach terminal states
- Delete only after jobs complete
- Fail safely if timeout reached (leave DISABLED, don't delete)
- Consider in-place tag updates as quick win
## Reproduction Steps
**1. Create initial stack with fixed JobQueue name:**
```yaml
AWSTemplateFormatVersion: '2010-09-09'
Resources:
ComputeEnvironment:
Type: AWS::Batch::ComputeEnvironment
Properties:
Type: MANAGED
ComputeResources:
Type: FARGATE
MaxvCpus: 256
Subnets:
- !Ref Subnet
SecurityGroupIds:
- !Ref SecurityGroup
State: ENABLED
JobQueue:
Type: AWS::Batch::JobQueue
Properties:
JobQueueName: my-fixed-queue-name # Fixed name
Priority: 1
State: ENABLED
ComputeEnvironmentOrder:
- Order: 1
ComputeEnvironment: !Ref ComputeEnvironment
Tags:
Environment: dev
VPC:
Type: AWS::EC2::VPC
Properties:
CidrBlock: 10.0.0.0/16
Subnet:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref VPC
CidrBlock: 10.0.1.0/24
SecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: Batch compute environment
VpcId: !Ref VPC
```
**2. Update the stack to trigger replacement (change tags):**
```yaml
# Change the Tags property:
Tags:
Environment: prod # Changed from dev
Owner: team-a # Added new tag
```
**3. Observe the error:**
- Stack update fails with `ConflictException: Object already exists (Status Code: 409)`
- CloudFormation attempts to create new queue with name `my-fixed-queue-name` before deleting old one
**4. To test with running jobs:**
- Submit jobs to the queue before step 2
- Observe that workarounds (like renaming) interrupt running jobs
Contributor guide
Research direction
The issue names no repository files, tests, or implementation entry points. Start with the AWS::Batch::JobQueue documentation and the referenced UpdateJobQueue, ListJobs, TagResource, and UntagResource APIs; done requires an agreed and validated way to replace or update a fixed-name queue without interrupting running jobs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws
- Domain
- cloud, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100