aws / aws/aws-toolkit-azure-devops

Use s3ObjectKey or randomize S3 object name when uploading Cloudformation template

Open
#524 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
258
Forks
114
PR merge metrics
No merged PRs in 30d

Description

**Is your feature request related to a problem? Please describe.**

I'm trying to use a "Common" Azure devops asset s3 bucket to store (temporarily) all CF template when I use the task `CloudFormationCreateOrUpdateStack@1`

to avoid creating too many buckets and all, I created a shared bucket in the CICD AWS account and shared it with my ORG, and then each account Azure Devops deployment role has read/write access to teh bucket to temporarily use it to store the CF template if required...

This makes my task look like this:
```
- task: CloudFormationCreateOrUpdateStack@1
name: DeployTransitGateway
displayName: "Deploy Transit Gateway"
inputs:
awsCredentials: 'AzureDevopsSecurityDeploymentRole'
regionName: 'ap-southeast-2'
stackName: 'transit-gateway'
templateSource: 'file'
templateFile: 'templates/transit-gateway.yaml'
templateParametersFile: 'configs/ap-southeast-2/transit-gateway.yaml'
s3BucketName: 'azuredevops-assets-xxxxxxxxx-ap-southeast-2'
s3ObjectKey: '437044670697/ap-southeast-2/transit-gateway.yaml'
```

Sadly, with this model, the upload of the CF template does work in the specified bucket, but uses the "basename" of the file to upload as the S3 object Key...

this poses a risk that someone / something, may use the same "template name" in a Different Azure Devops project / repository, and may break S3/CF deployment due to a race condition:

- Pipeline 1 uploads transit-gateway.yaml
- Pipeline 2 (Project B) uploads transit-gateway.yaml
- Pipline 1 tries to create/update stack using S3 object "transit-gateway.yaml" (which now points to pipeline 2 project file, so it's likely to break).

**Describe the solution you'd like**

I have 2 options in mind:
* Either randomly generate a checksum / UUID of the template to generate the S3 object Key.
* Use the `s3ObjectKey` parameter as the upload key name to override the default of the task during upload.

**Describe alternatives you've considered**

Yes - I know I could add a pre-task to do the S3 Upload to whatever path I need and then change the `CloudFormationCreateOrUpdateStack@1` task to set `templateSource: s3` and specify the key. But this is just too much work at this time for something that `CloudFormationCreateOrUpdateStack@1` almost does.

**Additional context**

Contributor guide

Open the contributing guide

Research direction

Start at the implementation of the CloudFormationCreateOrUpdateStack@1 task and trace how a local template is uploaded when s3BucketName is set. Check how s3ObjectKey is handled during upload; done means concurrent pipelines cannot overwrite each other's template object and the subsequent CloudFormation deployment uses the intended object.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, azure, typescript
Domain
cloud, devops
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.