Align output with `go test` to facilitate integration with editors

Open
#267 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
30/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Stale
Tech stack
go

Research direction

Start by comparing testify's failure output with the standard library's go test format and review the error messages produced through Fail. Use the vim-go regex and the examples in the issue as acceptance criteria; done means failures emit a single-line file:line summary without requiring a flag or environment variable.

Written by the indexing model from the issue text.

Description

enhancement enhancement: output format input needed

From PR #254:

Best thing to do to facilitate integration into tools like vim-go would be to align with the output from go test. Stdlib prints a single line per error as file.go:line: message.

As an example, vim-go has the following regex: let tokens = matchlist(line, '^\s*\(.\{-}\):\(\d\+\):\s*\(.*\)'). They have some logic to figure out the path of the file.

In order to align with go test and get our errors picked up by vim-go we need to output a single line of text matching the regex and showing a one-line summary.

e.g:

--- FAIL: TestExample (0.00s)
main_test.go:18: values should be equal

        Error Trace:    main_test.go:18
        Error:          Not equal: []string{"a", "b", "x"} (expected)
                                != []string{"a", "b", "y", "d"} (actual)

                        Diff:
                        --- Expected
                        +++ Actual
                        @@ -1,5 +1,6 @@
                        -([]string) (len=3 cap=3) {
                        +([]string) (len=4 cap=4) {
                          (string) (len=1) "a",
                          (string) (len=1) "b",
                        - (string) (len=1) "x"
                        + (string) (len=1) "y",
                        + (string) (len=1) "d"
                         }

However this would require reviewing all the error messages from testify and might require a breaking to include the one-liner error argument in Fail.

In any case, it's better to avoid adding the path. Plugins would already have logic to figure out the path based on the stdlib output. Adding the path can pollute the output considerably and make it less readable.

Furthermore, if we were to do it, I believe we should have it enabled by default rather than hiding it behind a flag or an env variable. That way it'll be integrated by the tool's default go test execution rather than require special configuration to have testify place nicely with editor plugins.

Dominant language
Go
Stars
26.2k
Forks
1.9k
Avg merge
2d 4h
Merged PRs (30d)
2

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from stretchr/testify

All issues in stretchr/testify

Similar issues

More Go issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.