GoogleContainerTools / GoogleContainerTools/skaffold

Support custom CLI tool for formatting logs

Open
#8,130 3 comments 0 reactions 0 assignees View on GitHub
area/cli area/logging kind/feature-request kind/todo priority/p3
Dominant language
Go
Stars
15.9k
Forks
1.7k
Avg merge
3d 9h
Merged PRs (30d)
10

Description

Some of our services do structural logging in JSON. For development, we pipe stdout/stderr to a CLI tool like [bunyan](https://github.com/trentm/node-bunyan) or [pino-pretty](https://github.com/pinojs/pino-pretty), which parses the JSON and prints human readable log lines instead.

I would like to do the same when using `skaffold dev`, but as skaffold prefixes lines and outputs the logs of multiple services, this is not straight forward.

### Expected behavior

Skaffold yaml:
```yaml
[...]
logs:
formatters:
SERVICE1:
stdout: bunyan
stderr: bunyan
SERVICE2:
stdout: pino -i v
```

Semantics:
- Capture stdout of SERVICE1 and pass it to an instance of `bunyan`. Capture the output of bunyan and treat it as if it was the original output of that service. In particular, prefix it as configured.
- Capture stderr of SERVICE1 and pass it to a second instance of `bunyan`. Analogously to above, treat the output as if it was the original stderr output of SERVICE1.
- For SERVICE2, transform only stdout, but not stderr. Pass stdout to `pino -i v` and handle output like described above.
- For all output, pass through the colors (don't strip it out).

### Actual behavior

There is no way of specifying a formatter.

A work-around is setting the log prefix to the empty string and pass it to pino. pino is smart enough to detect if a line is JSON and outputs non JSON lines verbatim. For bunyan, there is no such simple solution.

With the work-around, I see these drawbacks:
1. Text color is lost for non-json log lines and other output from skaffold.
2. There can only be one formatter that formats the output of all services, but it is often helpful to filter log levels or unwanted fields per service.
3. The log entries of services can't be prefixed, e.g. with service name.
4. Sometimes, e.g. in case of a container exiting, the output is prefixed even if the config doesn't allow prefixing (see example in the linked repo below).

### Information

- Skaffold version: v2.0.2
- Operating system: macOS Ventura
- Installed via: Homebrew
- Contents of skaffold.yaml:

```yaml
apiVersion: skaffold/v4beta1
kind: Config
metadata:
name: skaffold-example
deploy:
logs:
prefix: none
manifests:
rawYaml:
- deployment.yaml
```

### Steps to reproduce the behavior

1. Clone https://github.com/ebekebe/skaffold-example
2. `npm install -g pino-pretty`
3. `skaffold dev | pino-pretty`

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.