[deploy] Migrate ingress from nginx to traefik

Kubernetes support for ingress-nginx is expiring, so both qa and prod are being migrated.

This is a simple change for most projects, but because we use ingress-nginx in our local kubernetes deployments (dev, nuc, and laptop installer), we need to make sure the required change in qa/prod interferes with the local settings.

Current usage in The Combine

• traefik
• nginx
• ingress

Required helm updates (from Greg M in LTOps)

Added middleware for Traefik:

# Sets the maximum request body to 250MB
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: maxrequestbody-limit
spec:
  buffering:
    maxRequestBodyBytes: 262144000

maxrequestbody-limit above and below can be changed as desired.

Added helm config lines:

traefik.ingress.kubernetes.io/router.middlewares: thecombine-maxrequestbody-limit@kubernetescrd

traefik.ingress.kubernetes.io/router.tls: "true"

Updated helm value:

• ingressClass: change nginx to traefik

Todo

Untangle the qa/prod vs local ingress settings, possibly adding some conditional statements in deploy/helm/thecombine/ charts.

Less urgent follow-up:
Should we also migrate the local kubernetes cluster from nginx to traefik?