How to add custom output on test failure (for binary data comparison)
- Dominant language
- C++
- Stars
- 21.5k
- Forks
- 3.5k
- Avg merge
- 3d 16h
- Merged PRs (30d)
- 2
Description
## Description
I would like to port my tests away from Boost.Test and I have started investigating Catch. Everything looks good except I can't work out how to cleanly compare binary data with Catch. Because my code produces binary files, if a test fails because the output data is not what was expected, I need to see the offending data in the test output so I can work out what went wrong.
### Steps to reproduce
```
TEST_CASE("Binary data comparison") {
std::string a( "test\x00\x01\x02\x03ing!", 12 );
std::string b( "test\x03\x02\x01\x00ing!", 12 );
REQUIRE( a == b );
}
```
Output:
```
tests_catch.cpp:69: FAILED:
REQUIRE( a == b )
with expansion:
"testing!" == "testing!"
```
The binary data is not present in this format, and if it were, it would not be as clear as it could be.
### Extra information
Boost.Test lets me write a custom output function, so I wrote one that looks like this, using ANSI codes to colour the differing bytes:
```
test.cpp(566): error: in "test_xyz/test_open[xyz]": Example failure.
Exp: 74 65 73 74 00 01 02 03 69 6e 67 21 test....ing!
Got: 74 65 73 74 03 02 01 00 69 6e 67 21 test....ing!
```

Is it possible to do the same with Catch?
Contributor guide
Research direction
Reproduce the binary comparison in tests_catch.cpp using the shown REQUIRE(a == b) case and compare its output with the requested byte-oriented format. Read the existing assertion-reporting and custom-output documentation or entry points, if available, to determine whether this needs a supported extension or a new feature. Done means the framework can present the differing binary data clearly, including the relevant bytes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100