bottlerocket-os / bottlerocket-os/bottlerocket
Bottlerocket ignoring settings.container-registry.mirrors
- Dominant language
- Rust
- Stars
- 9.7k
- Forks
- 586
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 12
Description
It seems like bottlerocket is ignoring `settings.container-registry.mirrors`
We have configured Pull through cache in aws ECR and set the `settings.container-registry.mirrors` for quay.io, docker.io, ghcr.io and registry.k8s.io to use ECR Pull through cache, but bottlerocket instance is still pulling directly from the upstream.
I can clearly see that mirrors were set using `apiclient -u /settings` on the instance.
It is not even trying to pull from ECR pull-through cache.
Our current setting is smth like this:
```
[[settings.container-registry.mirrors]]
"registry" = "registry.k8s.io"
"endpoint" = [".dkr.ecr.us-east-1.amazonaws.com/k8s"]
[[settings.container-registry.mirrors]]
"registry" = "quay.io"
"endpoint" = [".dkr.ecr.us-east-1.amazonaws.com/quay"]
[[settings.container-registry.mirrors]]
"registry" = "docker.io"
"endpoint" = [".dkr.ecr.us-east-1.amazonaws.com/dockerhub"]
[[settings.container-registry.mirrors]]
"registry" = "ghcr.io"
"endpoint" = [".dkr.ecr.us-east-1.amazonaws.com/github"]
```
**Image I'm using:**
bottlerocket-aws-k8s-1.27-x86_64-v1.16.1
**What I expected to happen:**
Pull through ECR cache for images in quay.io, docker.io, ghcr.io and registry.k8s.io registires.
**What actually happened:**
Pulling directly from upstream. Not even trying to pull from the ECR cache
**How to reproduce the problem:**
- Configure "Pull through cache" in ECR
- Set `settings.container-registry.mirrors` in your bottlerocket instance using UserData.
- Check that the setting is set by connecting to the instance either by ssh or Session Manager and run `apiclient -u /settings`
- Let your node in EKS pull any image from quay.io, docker.io, ghcr.io or registry.k8s.io registries.
- See containerd logs or k8s events for the deployed application
Contributor guide
Research direction
Start by reproducing the issue with the listed settings and verify them using `apiclient -u /settings`. Inspect containerd logs and Kubernetes events while pulling images from the named registries; done means the configured ECR pull-through cache is used instead of the upstream registry.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, kubernetes, linux
- Domain
- infrastructure, operating-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100