(lambda): `lambda.DockerImageFunction` can not be used alongside `containerd` to utilize cache storage backends
- Dominant language
- TypeScript
- Stars
- 12.9k
- Forks
- 4.6k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 83
Description
### Describe the bug
## Context:
CDK provides the ability to utilize Docker cache storage backends as per [the docs](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_lambda.AssetImageCodeProps.html#cachefrom).
In order to make use of cache storage backends such as a local directory or external registry, it is a requirement to enable [containerd image store](https://docs.docker.com/desktop/containerd/) in Docker.
Once containerd has been enabled, you are able to populate the build cache parameters within CDK, and subsequent builds on different CI runners would all utilise the same backend cache, drastically improving the CDK build and deployment process.
## Bug
Once `containerd` has been enabled, any Lambda function utilizing the Docker runtime fails to deploy.
Lambda code:
```
new lambda.DockerImageFunction(
this,
"LambdaFn",
{
functionName: "containerd-function",
code: lambda.DockerImageCode.fromImageAsset(__dirname + "/lambda"),
}
);
```
Errror message:
> UPDATE_ROLLBACK_COMPLETE: Resource handler returned message: "The image manifest or layer media type for the source image ... is not supported. (Service: Lambda, Status Code: 400, Request ID: ...)"
## A likely resolution:
As per a similar issue noted on https://github.com/aws/aws-cdk/issues/30258 - The only workaround for this is to currently use `BUILDX_NO_DEFAULT_ATTESTATIONS=1 cdk deploy`
It would be better if CDK explicitly adds `--provenance=false` in its calls to docker buildx, or provides the ability to do so.
See https://docs.docker.com/reference/cli/docker/buildx/build/#provenance and https://docs.docker.com/build/attestations/attestation-storage/
### Regression Issue
- [ ] Select this option if this issue appears to be a regression.
### Last Known Working CDK Version
n/a
### Expected Behavior
I would expect to be able to utilize the documented cache backends feature within CDK when deploying Lambda functions that utilises a container image. These two CDK features are not able to work together.
### Current Behavior
Lambda fails to create or update the Lambda function with the following error message:
> The image manifest or layer media type for the source image is not supported
### Reproduction Steps
1. Enable containerd in Docker
2. Create a Lambda function in CDK using:
```
new lambda.DockerImageFunction(
this,
"LambdaFn",
{
functionName: "containerd-function",
code: lambda.DockerImageCode.fromImageAsset(__dirname + "/lambda"),
}
);
```
3. Create a Dockerfile using:
```
FROM --platform=linux/amd64 public.ecr.aws/lambda/python:3.12
COPY handler.py ./
CMD [ "handler.handler" ]
```
4. Create a handler.py using:
```
def handler(event, context):
return {"success": True}
```
Run `cdk deploy` to deploy the function.
### Possible Solution
Prefixing `BUILDX_NO_DEFAULT_ATTESTATIONS=1` to `cdk deploy` works around this issue.
### Additional Information/Context
_No response_
### CDK CLI Version
2.160.0 (build 7a8ae02)
### Framework Version
_No response_
### Node.js Version
v18.17.1
### OS
MacOS 14.6
### Language
TypeScript
### Language Version
_No response_
### Other information
_No response_
Contributor guide
Research direction
Start at lambda.DockerImageCode.fromImageAsset and trace the Docker buildx invocation used for image assets, then compare its output when containerd is enabled. Verify the change with the Dockerfile and handler.py reproduction, and confirm that cache backends work while Lambda accepts the resulting image manifest without BUILDX_NO_DEFAULT_ATTESTATIONS.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, docker, typescript
- Domain
- cloud, devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100