tilt-dev / tilt-dev/tilt

minikube detection heuristic does not work when embedding certificates on non-default profile

Open
#5,206 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Go
Stars
10.1k
Forks
413
Avg merge
1d 10h
Merged PRs (30d)
6

Description

I hope this is not a feature request, but I put it as a bug report cause it's existing functionality in tilt.

Expected Behavior

tilt should be able to detect when I'm running in minikube, even if I've enabled the option to embed certificates.

Current Behavior

tilt detects my minikube cluster as prod and I have to work around it with allow_k8s_contexts and other settings.

Steps to Reproduce
  1. minikube start --embed-certs -p test123
  2. tilt up

tilt will refuse to start, giving

Initial Build • (Tiltfile)
Loading Tiltfile at: /private/tmp/tilt_test/Tiltfile
Stop! test123 might be production.
If you're sure you want to deploy there, add:
allow_k8s_contexts('test123')
to your Tiltfile. Otherwise, switch k8s contexts and restart Tilt.

A really quick set of files for reproducing this is in https://github.com/Gibstick/tilt-minikube-heuristic-bug

Workaround

Go into the minikube profile's config.json and re-disable certificate embedding with EmbedCerts: false, or use a minikube profile name that is prefixed with minikube-.

Context
tilt doctor Output
$ tilt doctor
Tilt: v0.23.0, built 2021-11-12
System: darwin-amd64
---
Docker
- Host: Error: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
---
Kubernetes
- Env: unknown
- Context: delete-me-tilt-bug-report-gibstick
- Cluster Name: delete-me-tilt-bug-report-gibstick
- Namespace: default
- Container Runtime: docker
- Version: v1.22.3
- Cluster Local Registry: none
---
Thanks for seeing the Tilt Doctor!
Please send the info above when filing bug reports. 💗
Other notes

I asked about this in the tilt channel in the Kubernetes slack and one of your developers helpfully pointed to this part of the code: https://github.com/tilt-dev/tilt/blob/5b01cd34b0174b58b590343edac3d9c60d0c0ab6/internal/k8s/env.go#L138-L142

Here, it's expecting the kubernetes cluster config to be something like

- cluster:
    certificate-authority: /Users/charlie/.minikube/ca.crt
    extensions:
    - extension:
        last-update: Fri, 19 Nov 2021 10:20:35 EST
        provider: minikube.sigs.k8s.io
        version: v1.24.0
      name: cluster_info
    server: https://192.168.64.8:8443
  name: foobar

However, when using --embed-certs, it changes to

- cluster:
    certificate-authority-data: $BASE64_ENCODED_CERTIFICATE
    extensions:
    - extension:
        last-update: Fri, 19 Nov 2021 10:20:35 EST
        provider: minikube.sigs.k8s.io
        version: v1.24.0
      name: cluster_info
    server: https://192.168.64.8:8443
  name: foobar
About Your Use Case

I use minikube profiles because I work on multiple independent projects. All of those use tilt. I was following the instructions from https://minikube.sigs.k8s.io/docs/handbook/untrusted_certs/ because I needed to test something with my own root certificates. After I started minikube with --embed-certs, the issue started happening.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with internal/k8s/env.go around lines 138-142, then reproduce with minikube start --embed-certs -p test123 and tilt up. Compare the kubeconfig forms described in the issue and verify that Tilt recognizes the non-default Minikube profile instead of treating it as production.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
devops, infrastructure
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.