aws / aws/aws-cdk

pipelines: BundlingOptions Docker Login

Open
#25,808 6 comments 0 reactions 0 assignees View on GitHub
@aws-cdk/aws-codepipeline effort/medium feature-request p2
Dominant language
TypeScript
Stars
12.9k
Forks
4.6k
Avg merge
2d 3h
Merged PRs (30d)
83

Description

### Describe the bug

We are using `CodePipeline` in which we correctly specified our Docker Hub secret. It is correctly retrieved by the codebuild project. In this pipeline, we have a stage in which we are building a stack that has an `aws_s3_assets.s3_assets` resource. In this resource, we use `BundlingOptions' like this:
```
bundling=BundlingOptions(
image=DockerImage.from_registry("python:3.10"),
command=[
"sh",
"-c",
"""
pip install poetry;
],
user="root",
),
```

However, pulling the image `python:3.10` still results in a rate-limit issue. It seems like it's not using the specified Docker Hub secret we specified in the pipeline level. I verified using some command that we still haven't reached our account's rate limit.

Our pipeline looks like this:

```
pipeline = CodePipeline(
self,
"DataPipeline",
pipeline_name="DataPipeline",
docker_credentials=[DockerCredential.docker_hub(cdk.aws_secretsmanager.Secret.from_secret_name_v2(scope, "DockerHubSecret", secret_name="DockerHubSecret"))],
synth=ShellStep(
"Synth",
input=github_source,
commands=[
"pip install poetry",
"cd cdk",
"poetry install --no-dev",
],
primary_output_directory="./cdk/cdk.out",
),
code_build_defaults=CodeBuildOptions(role_policy=self._get_read_only_access_statements()),
docker_enabled_for_synth=True,
)
```

to which we add the stage that creates the `s3_assets`.

### Expected Behavior

Use the specified Docker Hub secrets then login to avoid the rate limit issue.

### Current Behavior

Unable to find image 'python:3.10' locally
266 | docker: Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit.

### Reproduction Steps

N/A

### Possible Solution

_No response_

### Additional Information/Context

_No response_

### CDK CLI Version

2.81.0

### Framework Version

_No response_

### Node.js Version

v16.17.0

### OS

macOS Ventura 13.3.1

### Language

Python

### Language Version

_No response_

### Other information

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by tracing how CodePipeline's docker_credentials and docker_enabled_for_synth reach BundlingOptions when aws_s3_assets.s3_assets pulls its image. Reproduce the failure with the shown Python configuration and verify that authenticated Docker Hub pulls are used instead of anonymous pulls.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, docker, python, typescript
Domain
ci-cd, cloud, devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.