coreos / coreos/coreos-assembler

kola: `--multiply` on a test with subtests doesn't work

Open
#3,873 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
393
Forks
193
Avg merge
17h 6m
Merged PRs (30d)
5

Description

When calling kolet the test name gets passed in mangled. I was able to see this with a small patch:

$ git diff mantle/kola/cluster/cluster.go
diff --git a/mantle/kola/cluster/cluster.go b/mantle/kola/cluster/cluster.go
index 24f3bce03..ce1898628 100644
--- a/mantle/kola/cluster/cluster.go
+++ b/mantle/kola/cluster/cluster.go
@@ -75,6 +75,7 @@ func (t *TestCluster) RunLogged(name string, f func(c TestCluster)) bool {
 // RunNative runs a registered NativeFunc on a remote machine
 func (t *TestCluster) RunNative(funcName string, m platform.Machine) bool {
        command := fmt.Sprintf("./kolet run %q %q", t.H.Name(), funcName)
+       t.Logf("command is %s", command)
        return t.Run(funcName, func(c TestCluster) {
                client, err := m.SSHClient()
                if err != nil {

and then running a simple --multiply=2 basic:

kola -p qemu run --multiply=2 basic --output-dir tmp/kola                                                             
⏭️   Skipping kola test pattern "fcos.internet":           
  👉 https://github.com/coreos/coreos-assembler/pull/1478                                                             
  👉 https://github.com/coreos/coreos-assembler/pull/1478                                                             
⏭️   Skipping kola test pattern "podman.workflow":                                                                    
  👉 https://github.com/coreos/coreos-assembler/pull/1478                                                             
🕒  Snoozing kola test pattern "ext.config.kdump.crash" until Sep 16 2024                                             
  👉 https://github.com/coreos/fedora-coreos-tracker/issues/1791                                                      
=== RUN   basic0                                           
=== RUN   basic1                                           
=== RUN   basic0/ReadOnly                                  
=== RUN   basic0/Useradd                                   
=== RUN   basic0/MachineID                                 
=== RUN   basic0/FCOSGrowpart                              
=== RUN   basic0/PortSSH                                   
=== RUN   basic0/DbusPerms                                 
=== RUN   basic0/ServicesActive                            
--- FAIL: basic0 (27.21s)                                  
        cluster.go:78: command is ./kolet run "basic0" "ReadOnly"                                                     
        cluster.go:96: kolet:                              
Usage:                                                     
  kolet run [test] [func] [flags]                          
  kolet run [command]                                      

Available Commands:                                        
  basic 

Note that kolet should have been called with ./kolet run "basic" "ReadOnly" but the multiplied test name got the appended 0, which appears to be breaking things.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in mantle/kola/cluster/cluster.go at TestCluster.RunNative and trace how multiplied test names are passed to the kolet run command. Reproduce with kola -p qemu run --multiply=2 basic; done means the remote command keeps the base test name while passing the subtest function correctly.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
cli, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.