bottlerocket-os / bottlerocket-os/bottlerocket
ecr-credential-provider: use custom AWS_PROFILE
- Dominant language
- Rust
- Stars
- 9.7k
- Forks
- 586
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 11
Description
**Image I'm using:**
```
apiclient get os
{
"os": {
"arch": "x86_64",
"build_id": "7452c37e",
"pretty_name": "Bottlerocket OS 1.18.0 (aws-k8s-1.28)",
"variant_id": "aws-k8s-1.28",
"version_id": "1.18.0"
}
}
```
**What I expected to happen:**
Setting `AWS_PROFILE` environment variable for `ecr-credential-provider` configures the AWS profile used by the `ecr-crendentials-provider`.
**What actually happened:**
The `AWS_PROFILE` environment variable is set twice: once with the configured value, once with value `default`.
**How to reproduce the problem:**
Add an AWS profile that is used by `ecr-credential-provider`:
```
[settings.aws]
config = "W3Byb2ZpbGUgZWNyXQpyb2xlX2FybiA9IGFybjphd3M6aWFtOjo8QUNDT1VOVF9JRD46cm9sZS9lY3ItcmVhZC1vbmx5CmNyZWRlbnRpYWxfc291cmNlID0gRWMySW5zdGFuY2VNZXRhZGF0YQo="
```
The `config` value base64-decoded equals to
```
[profile ecr]
role_arn = arn:aws:iam:::role/ecr-read-only
credential_source = Ec2InstanceMetadata
```
Set the `AWS_PROFILE` using `apiclient`:
```
[ssm-user@control]$ enter-admin-container
[root@admin]# apiclient set --json '{"settings":{"kubernetes":{"credential-providers":{"ecr-credential-provider":{"enabled":true,"image-patterns":["*.dkr.ecr.*.amaonaws.com"],"environment":{"AWS_PROFILE":"ecr"}}}}}}'
[root@admin]# cat /.bottlerocket/rootfs/etc/kubernetes/kubelet/credential-provider-config.yaml
apiVersion: kubelet.config.k8s.io/v1
kind: CredentialProviderConfig
providers:
- name: ecr-credential-provider
matchImages:
- "*.dkr.ecr.*.amaonaws.com"
defaultCacheDuration: "12h"
apiVersion: credentialprovider.kubelet.k8s.io/v1
env:
- name: AWS_PROFILE
value: 'ecr'
- name: HOME
value: '/root'
- name: AWS_PROFILE
value: 'default'
```
Also tried the following `userData` with Karpenter but that doesn't seem to be picked up. Will file [an issue](https://github.com/aws/karpenter-provider-aws/issues/5507) with Karpenter.
```
[settings.aws]
config = "W3Byb2ZpbGUgZWNyXQpyb2xlX2FybiA9IGFybjphd3M6aWFtOjo8QUNDT1VOVF9JRD46cm9sZS9lY3ItcmVhZC1vbmx5CmNyZWRlbnRpYWxfc291cmNlID0gRWMySW5zdGFuY2VNZXRhZGF0YQo="
[settings.kubernetes.credential-providers.ecr-credential-provider]
enabled = true
image-patterns = ["*.dkr.ecr.*.amazonaws.com"]
[settings.kubernetes.credential-providers.ecr-credential-provider.environment]
AWS_PROFILE = "ecr"
```
**Additional information:**
We have an EKS cluster in AWS account A and want it to pull images from the ECR registry of AWS account B. We try to configure the `ecr-credential-provider` to use an AWS profile that assumes a role of account B that has readonly access to the ECR registry of account B.
I've seen we can also configure [settings.aws.profile](https://bottlerocket.dev/en/os/1.18.x/api/settings/aws/) and set that (in this case) to `ecr`. This seems to work but does that mean everything that needs AWS services are using the `ecr` profile? We want to restrict ONLY the `ecr-credential-provider` to the `ecr` role/profile.
```
[settings.aws]
config = "W3Byb2ZpbGUgZWNyXQpyb2xlX2FybiA9IGFybjphd3M6aWFtOjo8QUNDT1VOVF9JRD46cm9sZS9lY3ItcmVhZC1vbmx5CmNyZWRlbnRpYWxfc291cmNlID0gRWMySW5zdGFuY2VNZXRhZGF0YQo="
profile = ecr
```
Contributor guide
Assessment
This issue has not been assessed yet.