aws / aws/aws-cdk

aws_s3_deployment: Files deleted when deployment bucket is used in sequence

Open
#30,871 4 comments 1 reaction 0 assignees View on GitHub
@aws-cdk/aws-s3-deployment bug effort/small p2
Dominant language
TypeScript
Stars
12.9k
Forks
4.6k
Avg merge
2d 3h
Merged PRs (30d)
83

Description

### Describe the bug

I have a stack for which I wish to deploy two scripts on S3.

I have two files under `../assets` which I want both deployed at the top level of the created `AssetBucket`. What I'm observing instead is that only one of the files ends up being deployed, and it seems like it's random, sometimes the corresponding to `SINGLE_ENTRYPOINT_FILENAME` is deployed, sometimes `ENTRYPOINT_FILENAME`.

Not sure if there's some sort of race condition going on.

### Expected Behavior

Both files should be deployed

### Current Behavior

Only one of the resource files actually ends up on S3

### Reproduction Steps

Here's how I'm trying to accomplish that:

```python
from aws_cdk import (
Stack,
aws_s3_deployment as s3_deploy,
)

DIRNAME = os.path.dirname(__file__)

class EntrypointAssetStack(Stack):
"""Creates the needed resources for the entrypoint assets on S3."""

def __init__(
self,
scope: Construct,
construct_id: str,
**kwargs,
) -> None:
super().__init__(scope, construct_id, **kwargs)

# Create an asset bucket where we upload the entry point script
# TODO: Trying to add removal policy here messes up bucket policy downstream, can we fix this?
asset_bucket = s3.Bucket(
self,
"AssetBucket",
# removal_policy=RemovalPolicy.DESTROY,
# auto_delete_objects=True,
block_public_access=s3.BlockPublicAccess.BLOCK_ALL,
versioned=True,
)

# Omitted a point where I create a policy for the asset bucket, asset_bucket_policy

entrypoint_deployment = s3_deploy.BucketDeployment(
self,
"DeployMultiEntryPoint",
destination_bucket=asset_bucket,
sources=[
s3_deploy.Source.asset(
path=os.path.join(DIRNAME, "..", "assets"),
exclude=["**", f"!{ENTRYPOINT_FILENAME}"],
)
],
retain_on_delete=True,
)
entrypoint_deployment.node.add_dependency(asset_bucket_policy)

single_entrypoint_deployment = s3_deploy.BucketDeployment(
self,
"DeploySingleWorkerEntryPoint",
destination_bucket=asset_bucket,
sources=[
s3_deploy.Source.asset(
path=os.path.join(DIRNAME, "..", "assets"),
exclude=["**", f"!{SINGLE_ENTRYPOINT_FILENAME}"],
)
],
retain_on_delete=True,
)
single_entrypoint_deployment.node.add_dependency(asset_bucket_policy)
```

### Possible Solution

_No response_

### Additional Information/Context

_No response_

### CDK CLI Version

2.149.0 (build c8e5924)

### Framework Version

_No response_

### Node.js Version

v18.18.2

### OS

Amazon Linux 2

### Language

Python

### Language Version

3.9.18

### Other information

_No response_

Contributor guide

Open the contributing guide

Research direction

Start with the aws_s3_deployment.BucketDeployment and Source.asset entry points described in the reproduction, then run the two sequential deployments against one bucket. Trace how each deployment handles existing objects and verify the behavior with a focused regression test; done means both entrypoint files remain in the bucket after deployment.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python
Domain
cloud, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.