microsoftgraph / microsoftgraph/microsoft-graph-comms-samples

ingress-tls is not READY after deploying TeamsRecordingBot to AKS

Open
#490 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
No language data
Stars
253
Forks
275
Avg merge
1d 5h
Merged PRs (30d)
1

Description

Describe the issue
I followed the steps described in aks.md file.

The cert-manager.bat was successful

D:\microsoft-graph-comms-samples\Samples\V1.0Samples\AksSamples\teams-recording-bot\deploy>cert-manager.bat
Creating cert-manager namespace
namespace/cert-manager created
Updating helm repo
"jetstack" already exists with the same configuration, skipping
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "ingress-nginx" chart repository
...Successfully got an update from the "jetstack" chart repository
Update Complete. ⎈Happy Helming!⎈
Installing cert-manager
NAME: cert-manager
LAST DEPLOYED: Wed Sep  1 23:48:41 2021
NAMESPACE: cert-manager
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
cert-manager has been deployed successfully!

In order to begin issuing certificates, you will need to set up a ClusterIssuer
or Issuer resource (for example, by creating a 'letsencrypt-staging' issuer).

More information on the different types of issuers and how to configure them
can be found in our documentation:

https://cert-manager.io/docs/configuration/

For information on how to configure cert-manager to automatically provision
Certificates for Ingress resources, take a look at the `ingress-shim`
documentation:

https://cert-manager.io/docs/usage/ingress/
Waiting for cert-manager to be ready
pod/cert-manager-676497949f-d9bnx condition met
pod/cert-manager-cainjector-864dc784dd-vt48q condition met
pod/cert-manager-webhook-5cb94c8949-827sr condition met
Installing cluster issuer
clusterissuer.cert-manager.io/letsencrypt created

The ingress-nginx.bat was also successful

D:\microsoft-graph-comms-samples\Samples\V1.0Samples\AksSamples\teams-recording-bot\deploy>ingress-nginx.bat
Creating ingress-nginx namespace
namespace/ingress-nginx created
Adding helm repositories
"ingress-nginx" already exists with the same configuration, skipping
Error: repo "https://kubernetes-charts.storage.googleapis.com/" is no longer available; try "https://charts.helm.sh/stable" instead
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "ingress-nginx" chart repository
...Successfully got an update from the "jetstack" chart repository
Update Complete. ⎈Happy Helming!⎈
Installing ingress-nginx
NAME: nginx-ingress
LAST DEPLOYED: Wed Sep  1 23:49:31 2021
NAMESPACE: ingress-nginx
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
The ingress-nginx controller has been installed.
It may take a few minutes for the LoadBalancer IP to be available.
You can watch the status by running 'kubectl --namespace ingress-nginx get services -o wide -w nginx-ingress-ingress-nginx-controller'

An example Ingress that makes use of the controller:

  apiVersion: networking.k8s.io/v1
  kind: Ingress
  metadata:
    annotations:
      kubernetes.io/ingress.class:
    name: example
    namespace: foo
  spec:
    rules:
      - host: www.example.com
        http:
          paths:
            - backend:
                serviceName: exampleService
                servicePort: 80
              path: /
    # This section is only required if TLS is to be enabled for the Ingress
    tls:
        - hosts:
            - www.example.com
          secretName: example-tls

If TLS is enabled for the Ingress, a Secret containing the certificate and key must also be provided:

  apiVersion: v1
  kind: Secret
  metadata:
    name: example-tls
    namespace: foo
  data:
    tls.crt: <base64 encoded cert>
    tls.key: <base64 encoded key>
  type: kubernetes.io/tls

However, when I run the verification steps, I see that the ingress-tls is NOT READY

kubectl get cert -n teams-recording-bot
NAME          READY   SECRET        AGE
ingress-tls   False   ingress-tls   4m43s

Additional Steps:
I followed the issue https://github.com/microsoftgraph/microsoft-graph-comms-samples/issues/456 and updated the cert-manager to v1.3.1 instead of v0.15.1. Still, I have the same issue.

Logs:

kubectl logs cert-manager-676497949f-d9bnx -n cert-manager
E0901 22:29:56.015774       1 sync.go:181] cert-manager/controller/challenges "msg"="propagation check failed" "error"="wrong status code '503', expected '200'" "dnsName"="poc.foc-poc-xxxxx.com" "resource_kind"="Challenge" "resource_name"="ingress-tls-ms6qc-1139532502-3132019148" "resource_namespace"="teams-recording-bot" "resource_version"="v1" "type"="HTTP-01"
I0901 22:30:06.016946       1 pod.go:58] cert-manager/controller/challenges/http01/selfCheck/http01/ensurePod "msg"="found one existing HTTP01 solver pod" "dnsName"="poc.foc-poc-xxxxx.com" "related_resource_kind"="Pod" "related_resource_name"="cm-acme-http-solver-6k622" "related_resource_namespace"="teams-recording-bot" "related_resource_version"="v1" "resource_kind"="Challenge" "resource_name"="ingress-tls-ms6qc-1139532502-3132019148" "resource_namespace"="teams-recording-bot" "resource_version"="v1" "type"="HTTP-01"
I0901 22:30:06.017025       1 service.go:43] cert-manager/controller/challenges/http01/selfCheck/http01/ensureService "msg"="found one existing HTTP01 solver Service for challenge resource" "dnsName"="poc.foc-poc-xxxxx.com" "related_resource_kind"="Service" "related_resource_name"="cm-acme-http-solver-7h7kc" "related_resource_namespace"="teams-recording-bot" "related_resource_version"="v1" "resource_kind"="Challenge" "resource_name"="ingress-tls-ms6qc-1139532502-3132019148" "resource_namespace"="teams-recording-bot" "resource_version"="v1" "type"="HTTP-01"
I0901 22:30:06.017079       1 ingress.go:92] cert-manager/controller/challenges/http01/selfCheck/http01/ensureIngress "msg"="found one existing HTTP01 solver ingress" "dnsName"="poc.foc-poc-xxxxx.com" "related_resource_kind"="Ingress" "related_resource_name"="cm-acme-http-solver-gwbgq" "related_resource_namespace"="teams-recording-bot" "related_resource_version"="v1beta1" "resource_kind"="Challenge" "resource_name"="ingress-tls-ms6qc-1139532502-3132019148" "resource_namespace"="teams-recording-bot" "resource_version"="v1" "type"="HTTP-01"
E0901 22:30:06.041717       1 sync.go:181] cert-manager/controller/challenges "msg"="propagation check failed" "error"="wrong status code '503', expected '200'" "dnsName"="poc.foc-poc-xxxxx.com" "resource_kind"="Challenge" "resource_name"="ingress-tls-ms6qc-1139532502-3132019148" "resource_namespace"="teams-recording-bot" "resource_version"="v1" "type"="HTTP-01"
kubectl logs cert-manager-webhook-5cb94c8949-827sr -n cert-manager
W0901 21:48:53.702907       1 client_config.go:608] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
W0901 21:48:53.704043       1 client_config.go:608] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0901 21:48:53.704230       1 webhook.go:69] cert-manager/webhook "msg"="using dynamic certificate generating using CA stored in Secret resource"  "secret_name"="cert-manager-webhook-ca" "secret_namespace"="cert-manager"
I0901 21:48:53.704622       1 server.go:148] cert-manager/webhook "msg"="listening for insecure healthz connections"  "address"=":6080"
I0901 21:48:53.705518       1 server.go:161] cert-manager/webhook "msg"="listening for secure connections"  "address"=":10250"
I0901 21:48:53.705633       1 server.go:187] cert-manager/webhook "msg"="registered pprof handlers"
I0901 21:48:53.707045       1 reflector.go:207] Starting reflector *v1.Secret (1m0s) from external/io_k8s_client_go/tools/cache/reflector.go:156
I0901 21:48:54.750026       1 dynamic_source.go:199] cert-manager/webhook "msg"="Updated serving TLS certificate"

Expected behavior
ingress-tls must be in READY state.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with Samples/V1.0Samples/AksSamples/teams-recording-bot/docs/deploy/aks.md and the deploy/cert-manager.bat and deploy/ingress-nginx.bat scripts. Reproduce the kubectl get cert result and inspect the cert-manager challenge logs for the HTTP-01 503; done means ingress-tls becomes READY and the challenge propagation check returns 200.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, kubernetes
Domain
cloud, devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.