aws / aws/aws-cdk

aws-s3-assets: Object keys don't preserve the full extension name for Python project

Open
#30,257 3 comments 0 reactions 0 assignees View on GitHub
@aws-cdk/aws-s3-assets bug effort/small p2
Dominant language
TypeScript
Stars
12.9k
Forks
4.6k
Avg merge
2d 3h
Merged PRs (30d)
83

Description

### Describe the bug

Same as #12699 but for Python project.

Neither simple upload nor bundling asset with command in Docker container maintain full extension (HASH.tar.gz -> HASH.gz).

"tar.gz" file is expected from SageMaker endpoint.

### Expected Behavior

I expect the s3 object key is ended with tar.gz

### Current Behavior

When using aws-s3-assets to bundle the asset HASH.tar.gz by executing a command in a Docker container and upload to S3 it is renamed to HASH.gz

Same occurs when passing HASH.tar.gz directly and is uploaded to S3 but renamed to HASH.gz

### Reproduction Steps

Reproduction Steps for Docker (same behavior when omitting bundling and passing path="path/my.tar.gz"

Code:
```python
import pathlib

import aws_cdk as cdk
from aws_cdk import (
Stack,
aws_s3_assets,
)
from constructs import Construct

HERE = pathlib.Path(__file__).parent

class CdkErrorStack(Stack):

def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
super().__init__(scope, construct_id, **kwargs)

self.models_s3_artifact = aws_s3_assets.Asset(
self,
"Models",
path=str(HERE),
bundling=cdk.BundlingOptions(
image=cdk.DockerImage.from_registry("debian"),
command=[
"bash", "-c", "tar -czvf /asset-output/dummy.tar.gz --files-from /dev/null"
],
output_type=cdk.BundlingOutput.ARCHIVED
)
)
cdk.CfnOutput(self, "TarURI", value=self.models_s3_artifact.s3_object_url)
```

It is possible to see that locally in `cdk.out` the tarball has correct extensions:
![error](https://github.com/aws/aws-cdk/assets/23408482/ee045ff3-0a6d-48fc-a212-c96c68d8cdb5)

But in S3 and CFN output is wrong:
![errors3](https://github.com/aws/aws-cdk/assets/23408482/0adcf47a-f209-412d-b07a-b809a8ed33f6)

### Possible Solution

_No response_

### Additional Information/Context

_No response_

### CDK CLI Version

2.142.1

### Framework Version

_No response_

### Node.js Version

v18.17.1

### OS

MacOSX

### Language

Python

### Language Version

Python 3.9.12

### Other information

_No response_

Contributor guide

Open the contributing guide

Research direction

Reproduce the issue with aws_s3_assets.Asset, BundlingOptions, and a tar.gz output, including the direct-upload case. Trace how the generated filename becomes the S3 object key and CloudFormation output; done when both bundled and direct tar.gz assets preserve the full tar.gz extension.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python
Domain
cloud, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.