After N timeouts, abort tests

If we're running CWL conformance tests and there is a 15 minutes timeout per test. there are 133 tests so this ends up in a 33 hours total timeout when running against an unhealthy cluster.

Is there a way to parametrize "after N timeouts/fails, abort the tests and mark them as failed" ?

Thanks,
Nico

Start by locating the CWL conformance test runner and how its 15-minute per-test timeout and failures are handled. Define where an N-timeout or failure threshold would be configured, then verify that reaching it stops the run and marks the remaining conformance tests as failed.

Written by the indexing model from the issue text.