live-update to a CRD without an image

The Problem

We're seeing more and more frameworks in Kubernetes that use CRDs that don't have images in them. Maybe the CRD builds its own image. Or maybe it forces you to choose from a limited set of pre-existing images.

As an example, let's say you want to live-update files to the pods created by Prometheus CRD: https://github.com/tilt-dev/tilt-example-frameworks/blob/master/prometheus/prometheus.yaml

Tilt's deploy pipeline works like this:

• tilt organizes kubernetes objects into groups
• tilt builds and inject images into the spec yaml, then apply the yaml to the cluster
• tilt tracks which pods belong to which objects
• tilt live-updates the containers that matches the images

Image names are an essential part of all these steps - to select objects to track, then to select containers to inject into.

Proposal

We mostly have a way to group and track objects without images. It looks like this:

k8s_resource(
  new_name='app',
  objects=[...], # object spec
  extra_pod_selectors=[{...}])

This mostly works, but is a bit manual (in a lot of cases, it's hard to figure out the labels in advance). It's also missing a way to select the containers to live-update. It maps roughly to the KubernetesDiscovery object (https://api.tilt.dev/kubernetes/kubernetes-discovery-v1alpha1.html).

I think, in the short term, we should add an API spec and function for live_update objects that "connects" to the underlying pod discovery.

live_update(
  name='app-live-update',
  kubernetes_discovery_name='app',
  steps=[...],
  containers=[...])

where the containers= argument is a system for selecting the containers to live-update, by name or by image.

As an aside - in the long-term, I'd love to have a tiltfile API that more closely matches the REST API, so you could imagine something like:

kubernetes_apply(name=..., yaml=..., ,kubernetes_discovery_template=...)
live_update(...)

That would save you the effort of manually grouping objects and setting labels. But i think we can punt on it for now.

Start with the existing k8s_resource API and its KubernetesDiscovery representation, then compare the proposed live_update API and container-selection requirements. Done means defining and implementing the API spec and function that connect live-update objects to pod discovery without requiring images.