godaddy / godaddy/kubernetes-client

CMD auth provider fails when using AWS roles

Open
#573 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
963
Forks
189
PR merge metrics
No merged PRs in 30d

Description

We've hit a corner case.

1) we have a `~/.kube/config` with this `users` section

```yaml
users:
- name: arn:aws:eks:us-east-1:...
user:
exec:
apiVersion: client.authentication.k8s.io/v1alpha1
args:
- --region
- us-east-1
- eks
- get-token
- --cluster-name
- my-eks-cluster
command: aws
env:
- name: AWS_PROFILE
value: role-profile
```

2) when we run the program that uses kubernetes-client, `process.env` includes `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` (we need them as env vars)

With these 2 preconditions, whenever we try to talk to the API server, we get an `Unauthorized` error. Reason is that `aws` cli will **ignore** AWS_PROFILE if `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` are env vars, so the token we get by running the command described in `~/.kube/config` is not valid to access the k8s cluster we created under role `role-profile`

Our workaround for now is to manually add

```yaml
- --profile
- role-profile
```

to the args of the command, so that it looks like this

```yaml
users:
- name: arn:aws:eks:us-east-1:...
user:
exec:
apiVersion: client.authentication.k8s.io/v1alpha1
args:
- --profile
- role-profile
- --region
- us-east-1
- eks
- get-token
- --cluster-name
- my-eks-cluster
command: aws
env:
- name: AWS_PROFILE
value: role-profile
```

"Manually" because the k8s config above was generated by `aws eks update-kubeconfig`

We also tried patching `cmd.js` this way

```diff
diff --git a/backends/request/auth-providers/cmd.js b/backends/request/auth-providers/cmd.js
index fd70052..6f5d548 100644
--- a/backends/request/auth-providers/cmd.js
+++ b/backends/request/auth-providers/cmd.js
@@ -32,6 +32,10 @@ module.exports = {
const args = config['cmd-args'].split(' ')
const cmdEnv = config['cmd-env']

+ if (cmd === 'aws' && args.indexOf('--profile') === -1 && cmdEnv.AWS_PROFILE) {
+ args.unshift(...['--profile', cmdEnv.AWS_PROFILE])
+ }
+
let output

if (process.platform === 'win32') {
```

it worked but it looked hacky, so we preferred the workaround (thus this issue, instead of a PR)

Versions:
- aws cli: 1.16.313
- kubernetes-client: 8.3.6

Contributor guide

Open the contributing guide

Research direction

Start in backends/request/auth-providers/cmd.js, especially the handling of cmd-args and cmd-env for exec authentication commands. Reproduce the AWS_PROFILE and credential environment variable combination, then verify that the configured AWS role is used without breaking commands that already specify --profile.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, javascript, node.js
Domain
authentication, cli, cloud
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.