matrix-org / matrix-org/complement

`gotestfmt` can mask errors: "BUG: Empty package name encountered."

Open
#821 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

good first issue help wanted
Dominant language
Go
Stars
99
Forks
72
Avg merge
4d 1h
Merged PRs (30d)
8

Description

This surfaced in https://github.com/matrix-org/complement/pull/820 (potentially due to the `go` version bump?).

We currently pipe all output from `go test -json ...` into the [gotestfmt](https://github.com/gotesttools/gotestfmt) tool:

https://github.com/matrix-org/complement/blob/9d4a0219ef7eac8efc96461e7198558de5e66d08/.github/workflows/ci.yaml#L121

If there's a problem with the test, it may emit something like:

```
Error: {"ImportPath":"github.com/matrix-org/complement/tests/csapi [github.com/matrix-org/complement/tests/csapi.test]","Action":"build-output","Output":"tests/csapi/ignored_users_test.go:55:11: non-constant format string in call to (*testing.common).Logf\n"}
```

instead of the usual:

```
{"Time":"2025-11-20T10:50:56.786423108Z","Action":"output","Package":"github.com/matrix-org/complement/tests","Test":"TestMSC4289PrivilegedRoomCreators_Upgrades","Output":" client.go:805: [CSAPI] POST hs1/_matrix/client/v3/join/!BDAhP0DZ5sVIHnOhxuAxN3v4kAMszvp0VaUAlsM2p-M =\u003e 200 OK (24.248184ms)\n"}
```

Note that the erroring log line doesn't contain a `Package` field.

`gotestfmt` chokes in this case, and outputs the cryptic error above. The only way to see the true error is to remove `| gotestfmt` from CI and re-run, grep'ing the logs for something like `"Output":"tests/`.

`gotestfmt` appears to be unmaintained since 2023. We may wish to fork upstream and fix the issue so such errors are presented cleanly, or choose a maintained alternative.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at .github/workflows/ci.yaml around line 121 and inspect how go test -json output is piped into gotestfmt. Re-run the workflow without gotestfmt to compare the underlying build error with the formatter output. Done means CI presents errors such as missing Package fields clearly instead of masking them with “Empty package name encountered.”

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, go
Domain
ci-cd, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.