GoogleContainerTools / GoogleContainerTools/container-structure-test

Inconsistent log levels for test names

Open
#471 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
2.5k
Forks
212
PR merge metrics
No merged PRs in 30d

Description

# Description
Different tests log their name at different log levels:
- `Info` level logging:
- File existence: https://github.com/GoogleContainerTools/container-structure-test/blob/main/pkg/types/v2/file_existence.go#L89
- file_content: https://github.com/GoogleContainerTools/container-structure-test/blob/main/pkg/types/v2/file_content.go#L60
- `Debug` level logging:
- command: https://github.com/GoogleContainerTools/container-structure-test/blob/main/pkg/types/v2/command.go#L84
- licenses: https://github.com/GoogleContainerTools/container-structure-test/blob/main/pkg/types/v2/licenses.go#L66
- Metadata: https://github.com/GoogleContainerTools/container-structure-test/blob/main/pkg/types/v2/metadata.go#L93

If `container-structure-test` runs with `--verbosity info` and `--output --test-report `, we can't see all the tests that are being run.

**Not knowing what tests will run is a possibility in complex build environments where the configuration for `container-structure-tests` is generated at test time.** Therefore it is helpful for all of this information to be available in a consistent manner without having to

I think it would make sense to make all the tests log their name at info level; this way it doesn't affect anyone was using `--verbosity debug`, whilst allowing those at `--verbosity info` to see all the tests being run.

# Replication
With the following `config.yml`:
```yaml
schemaVersion: "2.0.0"

fileExistenceTests:
- name: "sh existence"
path: "/bin/sh"

metadataTest:
entrypoint: []
```
Running with the flags ` info --image docker.io/library/alpine:latest --config config.yml --output junit --test-report out.xml` will give us the following results:
- with `--verbosity info`

```
$ container-structure-test test --verbosity info --image docker.io/library/alpine:latest --config config.yml --output junit --test-report out.xml
INFO[0000] container-structure-test &{Version:1.19.1 GitVersion: BuildDate:2024-07-22T18:56:16Z GoVersion:go1.22.5 Compiler:gc Platform:linux/amd64}
INFO[0000] File Existence Test: sh existence
```
- with `--verbosity debug`:

```
$ container-structure-test test --verbosity debug --image docker.io/library/alpine:latest --config config.yml --output junit --test-report out.xml
INFO[0000] container-structure-test &{Version:1.19.1 GitVersion: BuildDate:2024-07-22T18:56:16Z GoVersion:go1.22.5 Compiler:gc Platform:linux/amd64}
INFO[0000] File Existence Test: sh existence
DEBU[0000] Metadata Test
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.