(aws-s3): bucket policy fails to create when bucket:arn is not yet available
- Dominant language
- TypeScript
- Stars
- 12.9k
- Forks
- 4.6k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 83
Description
### Describe the bug
A dependency issue between S3 Buckets and Bucket Policies in the L2 Bucket class allows the Policy to access the arn of the bucket before it is available, causing the creation of the Bucket Policy to fail. Being a dependency issue, this is an intermittent issue and works correctly the vast majority of the time. When it fails, simply relaunching the stack usually works.
### Expected Behavior
The L2 Bucket construct should launch successfully every time.
### Current Behavior
testPolicy9D625504
CREATE_FAILED
Unable to retrieve Arn attribute for AWS::S3::Bucket, with error message Bucket not found
### Reproduction Steps
I created a simple CDK app with this code:
```
import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as s3 from 'aws-cdk-lib/aws-s3';
export class BucketPolicyDependencyStack extends cdk.Stack {
constructor(scope: Construct, id: string, props?: cdk.StackProps) {
super(scope, id, props);
new s3.Bucket(this, 'test', {
removalPolicy: cdk.RemovalPolicy.DESTROY,
autoDeleteObjects: true
})
}
}
```
I then set up a bash script that launched it 40 times, essentially simultaneously:
```
export constructs="
// Put any 30 values here, I just used 30 integers
"
for iteration in $constructs; do
export STACK_NAME=stresstest$iteration
cdk deploy -o stress$iteration --require-approval never &
done
```
On 1 of the 30 I saw the error I reference above.
### Possible Solution
If I am interpreting the behavior correctly, it seems that adding a Dependency on the Bucket to the BucketPolicy in the L2 Construct would prevent the Policy from trying to access the bucket before it is ready. Perhaps here? https://github.com/aws/aws-cdk/blob/3318a38a6092275d461ef3549f3b92cd0d040c18/packages/aws-cdk-lib/aws-s3/lib/bucket.ts#L651
### Additional Information/Context
We've seen it in several of our constructs (and newer versions of the CDK than what I cite below for the test above). Someone also mentioned they have seen it in aws-codepipline.
### CDK CLI Version
2.108.0
### Framework Version
2.108.0
### Node.js Version
20.9.0
### OS
MacOS Ventura 13.6.3
### Language
TypeScript
### Language Version
Typescript 5.2.2
### Other information
Versions cited are for the test I cited, but it's been seen in other versions as well.
Contributor guide
Research direction
Start in packages/aws-cdk-lib/aws-s3/lib/bucket.ts around line 651, where the issue suggests adding the dependency between the Bucket and BucketPolicy. Reproduce the concurrent deployment with the provided TypeScript stack and bash loop, then verify that repeated launches no longer fail with the missing Bucket Arn error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, typescript
- Domain
- cloud, infrastructure
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100