openshift / openshift/os

Consider adding kola test to check for Butane spec stabilization

Open
#1,336 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

jira lifecycle/frozen
Dominant language
Shell
Stars
135
Forks
131
Avg merge
4d 13h
Merged PRs (30d)
6

Description

Currently we don't have anything that ensures we stabilize a Butane openshift spec every OCP release cycle. Per discussion with @dustymabe, we could add a test so we don't forget to do this.

Butane is invoked by users, not by tools, and the user also selects the spec to be used. Also, Butane isn't in the OS or the release payload, nor does it have a downstream container image. As a result, figuring out what to check isn't completely trivial.

Proposal:

  1. Compute the OCP version from the RHCOS version.
  2. Generate an openshift Butane config with a non-experimental version matching the OCP version.
  3. Download the latest Butane from the mirror.
  4. See if the config transpiles. In Git master, it probably never will, assuming RHCOS is branched before Butane stabilizes the new spec. But in the release branch, it eventually will, late in the OCP development cycle.
  5. Use a snooze to disable the test until after the expected spec stabilization date. When tests fail on master, bump the snooze. When they fail on the branch, there's an actual problem.

This is all a bit awkward, though, since it would fail RHCOS builds solely to draw attention to an unrelated problem.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating existing kola tests and the source of the RHCOS version used by them. Evaluate the proposed OCP-version computation, Butane mirror download, transpilation check, and snooze behavior; done means the agreed test reliably distinguishes expected master failures from release-branch failures.

Written by the indexing model from the issue text.

Assessment

Tech stack
shell
Domain
testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.