GoogleContainerTools / GoogleContainerTools/skaffold

Improving "skaffold test" command output

Open
#6,571 0 comments 2 reactions 0 assignees View on GitHub
area/testing kind/feature-request kind/friction priority/p2 source/partnerships
Dominant language
Go
Stars
15.9k
Forks
1.7k
Avg merge
3d 9h
Merged PRs (30d)
10

Description

Currently, the `skaffold test` command runs all tests sequentially and stops on the first test failure. This experience can be relatively slow (running sequentially when parallelization may be possible) and potentially frustrating (stopping on the first test failure means that we don't have a comprehensive list of failing tests means we may have to run the command multiple times to catch all failures instead of running once and having a list of failures).

Further, the output of `skaffold test` does not report individual test success/failure via the event api, which means that in Cloud Code, the output of the test phase is not broken down by test. Instead the entire test phase either fails or passes with no easy indication of which test failed unless the user reads through the logs.

Note that in the image below, there is only a root `Test` node with no children, even though 2 tests are run:

Screen Shot 2021-09-07 at 2 15 24 PM

Accordingly, three proposed enhancements to the `skaffold test` command are:
1. Allow the tests to be run in parallel via a flag such as `--run-tests-in-parallel`, keeping sequential runs as the default
1. Run all tests to completion by default, potentially allowing the user to specify a `--fail-fast` or similar flag
1. Report individual test success/failure via the skaffold event api for consumption by Cloud Code or other clients

cc @ahmetb

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.