aws / aws/amazon-eks-pod-identity-webhook

SELF-HOSTED-SETUP: Certificate signed by unknown authority

Open
#131 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
691
Forks
203
Avg merge
4h 38m
Merged PRs (30d)
1

Description

**What happened**:
I'm trying to implement IRSA for a Self-hosted RKE2 kubernetes environment following the documentation here: [SELF_HOSTED_SETUP](https://github.com/aws/amazon-eks-pod-identity-webhook/blob/master/SELF_HOSTED_SETUP.md).

I've configured everything, however when I deploy a new awscli pod for testing I see the errors below.

***kube-apiserver logs***
>W1214 23:48:33.374702 1 dispatcher.go:170] Failed calling webhook, failing open pod-identity-webhook.amazonaws.com: failed calling webhook "pod-identity-webhook.amazonaws.com": Post "https://pod-identity-webhook.kube-system.svc:443/mutate?timeout=30s": x509: certificate signed by unknown authority
> E1214 23:48:33.375314 1 dispatcher.go:171] failed calling webhook "pod-identity-webhook.amazonaws.com": Post "https://pod-identity-webhook.kube-system.svc:443/mutate?timeout=30s": x509: certificate signed by unknown authority

***pod-identity-webhook logs***
>W1214 23:47:26.502020 1 client_config.go:552] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
>I1214 23:47:26.512708 1 store.go:63] Fetched secret: kube-system/pod-identity-webhook
>I1214 23:47:26.512961 1 main.go:195] Creating server
>I1214 23:47:26.513130 1 main.go:215] Listening on :9999 for metrics and healthz
>I1214 23:47:26.513242 1 main.go:209] Listening on :443
>2021/12/14 23:48:33 http: TLS handshake error from 10.42.0.0:47782: remote error: tls: bad certificate

Is there somewhere specific I should check or any files I can provide to help narrow down the issue?

I've attached the [kube-apiserver.yaml](https://github.com/aws/amazon-eks-pod-identity-webhook/files/7721288/kube-apiserver.txt) in case something in there looks off.

**What you expected to happen**:
AWS env variable to be injected into the awscli pod on creation.

**How to reproduce it (as minimally and precisely as possible)**:

Stand up an ([RKE2 Cluster](https://github.com/rancher/rke2)) and follow the [SELF_HOSTED_SETUP](https://github.com/aws/amazon-eks-pod-identity-webhook/blob/master/SELF_HOSTED_SETUP.md).

Terraform deployment for RKE2 also located [here](https://github.com/rancherfederal/rke2-aws-tf/tree/master/examples/cloud-enabled).

**Anything else we need to know?**:
The deployment files [here](https://github.com/aws/amazon-eks-pod-identity-webhook/tree/master/deploy) were modified to deploy into the kube-system namespace instead of the default one.

**Environment**:
- AWS Region: us-gov-west-1
- EKS Platform version (if using EKS, run `aws eks describe-cluster --name --query cluster.platformVersion`): N/A
- Kubernetes version (if using EKS, run `aws eks describe-cluster --name --query cluster.version`): v1.19.7+rke2r1"
- Webhook Version: v0.2.0

Contributor guide

Open the contributing guide

Research direction

Start with SELF_HOSTED_SETUP.md and the modified files under deploy, then compare them with the attached kube-apiserver.yaml and the webhook certificate-related logs. Reproduce the RKE2 setup and verify that the API server trusts the webhook certificate; done means the awscli pod is created with the expected AWS environment variables and no x509 errors appear.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, kubernetes, terraform
Domain
devops, infrastructure, security
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.