operator-framework / operator-framework/operator-sdk

Idea: Streamline scaffolded controller and e2e tests with Sawchain

Open
#7,085 2 comments 8 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

lifecycle/frozen
Dominant language
Go
Stars
7.7k
Forks
1.8k
PR merge metrics
No merged PRs in 30d

Description

Context

A fresh operator-sdk init + create api project scaffolds two test surfaces:

  • internal/controller/<kind>_controller_test.go — a Ginkgo skeleton on envtest + the controller-runtime client, stopping at a single Reconcile call with a // TODO(user): Add more specific assertions comment.
  • test/e2e/e2e_test.go — a Ginkgo suite driving everything through exec.Command("kubectl", ...) with stdout parsing.

Sawchain is a Go library (built on Chainsaw) with Gomega integration that works equally well for both: low-level integration tests against envtest and full e2e tests against a live cluster, using the same API on top of the same controller-runtime client. It offers:

  • Readable, declarative assertions — YAML describes the expected resource state instead of repetitive/imperative Get + field-by-field checks.
  • Partial matching with JMESPath — assert only the fields you care about, and bind values across assertions.
  • Ergonomic lifecycle helpersCreateAndWait, UpdateAndWait, DeleteAndWait bundle each write with an observability wait, so tests don't race against client-cache sync.
  • Mix-and-match — YAML-driven and struct-based logic coexist in the same test; pick whichever fits each assertion.

I wanted to share it as something that could simplify and enhance both scaffolded test surfaces.

Controller integration tests (internal/controller/...)

Any realistic test past the scaffolded TODO quickly grows into repetitive client.Get + field-by-field assertion loops.

Traditional pattern (simplified from a PodSet controller example):

podSet := &v1.PodSet{
    ObjectMeta: metav1.ObjectMeta{Name: "test-podset", Namespace: "default"},
    Spec: v1.PodSetSpec{
        Replicas: ptr.To(2),
        Template: v1.Template{
            Name:       "test-pod",
            Containers: []v1.Container{{Name: "test-app", Image: "test/app:v1"}},
        },
    },
}
Expect(k8sClient.Create(ctx, podSet)).To(Succeed())

Eventually(func() error {
    if err := k8sClient.Get(ctx,
        client.ObjectKeyFromObject(podSet), podSet); err != nil {
        return err
    }
    if len(podSet.Status.Pods) != 2 {
        return fmt.Errorf("expected 2 pods, got %d", len(podSet.Status.Pods))
    }
    return nil
}).Should(Succeed())

for _, podName := range podSet.Status.Pods {
    pod := &corev1.Pod{}
    Expect(k8sClient.Get(ctx, client.ObjectKey{Name: podName, Namespace: "default"}, pod)).To(Succeed())
    Expect(pod.Spec.Containers).To(HaveLen(1))
    Expect(pod.Spec.Containers[0].Name).To(Equal("test-app"))
    Expect(pod.Spec.Containers[0].Image).To(Equal("test/app:v1"))
}

With Sawchain:

podSet := &v1.PodSet{}
sc.CreateAndWait(ctx, podSet, `
    apiVersion: apps.example.com/v1
    kind: PodSet
    metadata:
      name: test-podset
      namespace: ($namespace)
    spec:
      replicas: 2
      template:
        name: test-pod
        containers:
        - name: test-app
          image: test/app:v1
`)

Eventually(sc.FetchSingleFunc(ctx, podSet)).Should(HaveField("Status.Pods", ConsistOf(
    "test-pod-0",
    "test-pod-1",
)))

for _, podName := range podSet.Status.Pods {
    Eventually(sc.CheckFunc(ctx, `
        apiVersion: v1
        kind: Pod
        metadata:
          name: ($name)
          namespace: ($namespace)
        spec:
          containers:
          - name: test-app
            image: test/app:v1
    `, map[string]any{"name": podName})).Should(Succeed())
}

CreateAndWait handles lifecycle, CheckFunc expresses declarative assertions with partial matching and JMESPath, and FetchSingleFunc polls state. There are also helpers like HaveStatusCondition, MatchYAML, and many more — check out the design overview for the full capabilities.

E2E tests (test/e2e/...)

The scaffolded e2e suite verifies cluster state by shelling out to kubectl and parsing the output: fragile, deeply coupled to kubectl's output shape, and hard to extend.

Scaffolded pattern (from e2e_test.go):

verifyCurlUp := func(g Gomega) {
    cmd := exec.Command("kubectl", "get", "pods", "curl-metrics",
        "-o", "jsonpath={.status.phase}",
        "-n", namespace)
    output, err := utils.Run(cmd)
    g.Expect(err).NotTo(HaveOccurred())
    g.Expect(output).To(Equal("Succeeded"), "curl pod in wrong status")
}
Eventually(verifyCurlUp, 5*time.Minute).Should(Succeed())

With Sawchain (same controller-runtime client, pointed at the live cluster):

Eventually(sc.CheckFunc(ctx, `
    apiVersion: v1
    kind: Pod
    metadata:
      name: curl-metrics
      namespace: ($namespace)
    status:
      phase: Succeeded
`, map[string]any{"namespace": namespace})).Should(Succeed())

The same pattern replaces the rest of the scaffold's kubectl-shell-out sprawl: namespace setup, pod polling, endpoint checks, token minting. No exec.Command, no stdout parsing, no jsonpath templates.

More

I would love to hear whether this kind of integration could be valuable. Happy to discuss further! 😄

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the scaffolding paths for operator-sdk init and create api, especially internal/controller/<kind>_controller_test.go and test/e2e/e2e_test.go. Compare the generated envtest and e2e flows with Sawchain's documented controller and live-cluster APIs. Done would require an agreed integration scope and acceptance criteria for replacing or augmenting both scaffolded test surfaces.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
developer-experience, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.