haskell / haskell/cabal

`cabal test` outputs stdout and stderr out of order

Open
#3,024 11 comments 0 reactions 0 assignees View on GitHub
type: enhancement
Dominant language
Haskell
Stars
1.7k
Forks
750
Avg merge
4d 3h
Merged PRs (30d)
28

Description

With https://github.com/hdgarrood/cabal-test-stderr, if you run `cabal test`, you get this:

```
Preprocessing test suite 'tests' for cabal-test-stderr-0.1.0.0...
Running 1 test suites...
Test suite tests: RUNNING...
oops, failed
lol
Test suite tests: FAIL
Test suite logged to: dist/test/cabal-test-stderr-0.1.0.0-tests.log
0 of 1 test suites (0 of 1 test cases) passed.
```

But if you run the test executable directly, you get this:

```
lol
oops, failed
```

That is, `lol` and `oops, failed` appear the other way around.

The source code is:

```
import Control.Monad
import System.IO
import System.Exit

main = do
putStrLn "lol"
hPutStrLn stderr "oops, failed"
exitFailure
```

I expected `cabal test` to produce text output in the same order as running the executable directly, even if the test executable outputs text to both stderr and stdout.

Possibly related: #1601. Being able to easily run the test suite directly would make this less of a problem.

Contributor guide

Open the contributing guide

Research direction

Reproduce the ordering difference with the Haskell program in the issue, comparing `cabal test` with running the test executable directly. Start at the `cabal test` path and the related discussion in #1601; done means stdout and stderr appear in the same order under Cabal as they do from the executable.

Written by the indexing model from the issue text.

Assessment

Tech stack
haskell
Domain
build-system, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.