aws / aws/aws-cli

(aws cloudformation package) add support for URIs from 3rd party artifact repositories

Open
#7,770 3 comments 1 reaction 0 assignees View on GitHub
cloudformation package-deploy feature-request p2
Dominant language
Python
Stars
17.3k
Forks
4.6k
Avg merge
1d 2h
Merged PRs (30d)
13

Description

### Describe the feature

The `aws cloudformation package` should optionally support the property values to be a 3rd party artifact repositories URI. When provided, the command will attempt to download the artifact from the provided URI, upload it to S3, and replace the property value with the new S3 URI.

### Use Case

In our CI/CD pipeline, it is common to download artifacts from an artifact repository (like Artifactory) just to upload it to S3 so it can be used by CloudFormation.

Let's take the template snippet below as an example.

```YAML
myFunction:
Type: AWS::Lambda::Function
Properties:
FunctionName: myFunction
Runtime: python3.8
Role: !GetAtt LambdaBasicExecutionRole.Arn
Handler: lambda_function.handler
Code: myArtifact.zip
```
At the moment, the CI/CD pipeline will have to download the artifact from the artifact repository, and store it in the same directory where the `package` command will be executed with the same name defined in the template. If there are any discrepancies, the artifact will not be found and the command will fail and ultimately so will the pipeline.

### Proposed Solution

It would be better developer experience if `package` command could support downloading the artifact from the provided URI and upload it to S3. The template snippet would look something like the one below:
```YAML
PythonFunction:
Type: AWS::Lambda::Function
Properties:
FunctionName: cfn-workshop-python-function
Description: Python Function to return specific TimeZone time
Runtime: python3.8
Role: !GetAtt LambdaBasicExecutionRole.Arn
Handler: lambda_function.handler
Code: https://artifactory.domain.com:443/artifactory/repository/path/myArtifact.zip
```

### Other Information

This could be supported by adding the logic below to the export method of the Resource class found [here](https://github.com/aws/aws-cli/blob/c05713eab7d994d3806c52b4c7dc30d99db24190/awscli/customizations/cloudformation/artifact_exporter.py#L234).
```python
temp_dir = None
if is_repository_url(property_value):
if self.support_artifact_repositories:
LOG.debug("Property is an artifact repository URI but --support-artifact-repositories was not provided.")
raise exceptions.ExportFailedError()
if not is_approved_repository_domain(property_value):
LOG.debug("Property is an artifact repository URI, but host is not found in the --approved-artifact-repository-domains list.")
raise exceptions.ExportFailedError()
temp_dir = download_artifact_to_temp_dir(property_value)
set_value_from_jmespath(resource_dict, self.PROPERTY_NAME, temp_dir)
```

In the example above, I added a conditions to check whether the support should be enabled and whether the URI provided is part of a list of approved domains, as I don't think we want the `package` command to download artifacts by default or from anywhere. With that in mind, I think we could add two optional parameters [here](https://github.com/aws/aws-cli/blob/c05713eab7d994d3806c52b4c7dc30d99db24190/awscli/customizations/cloudformation/package.py#L48), maybe something like:
```python
ARG_TABLE = [
{
'name': 'support-artifact-repositories',
'action': "store_true",
'help_text': (
'Indicates whether to support downloading artifacts from 3rd party artifact repositories.'
' Defaults to False.'
)
},
{
'name': 'approved-artifact-repository-domains',
'action': 'store',
'schema': {
'type': 'array',
'items': {
'type': 'string'
}
},
'default': [],
'help_text': (
'A list of approved domains where artifacts can be downloaded'
'to be uploaded to the Amazon S3 bucket. If not provided, all domains'
'will be denied.'
'Syntax: domain1.com domain2.com ...'
)
},
]
```
_Please note: This is just an example of how it could be done. I'm sure there are other questions to be considered when actually implementing it, like how to handle authentication or options to perform any actions on the downloaded artifact before uploading it._

### Acknowledgements

- [ ] I may be able to implement this feature request
- [ ] This feature might incur a breaking change

### CLI version used

aws-cli/2.10.4

### Environment details (OS name and version, etc.)

Python/3.11.2 Darwin/21.6.0 source/x86_64 prompt/off

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.