catchorg / catchorg/Catch2

Use CATCH_TRANSLATE_EXCEPTION for printing failures in *_THROWS_MATCHES

Open
#2,911 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
21.5k
Forks
3.5k
Avg merge
3d 16h
Merged PRs (30d)
2

Description

**Description**

Using `CATCH_TRANSLATE_EXCEPTION` is convenient, but sometimes we want to test the concrete type _and_ message. Using `*_THROWS_MATCHES` does not use the function provided by `CATCH_TRANSLATE_EXCEPTION` and just uses `std::exception::what()` (as documented). The output of `what()` prefixes the output of `describe()` without quotes or other annotation making the error harder to read and understand.

It may be that this is intentional and thus neither a bug nor a wanted feature. If so then I wonder if there is some way to customize the generated error messages that I may have missed in the docs.

**Additional context**

We're using version 3.6.0 here, but I will test 3.7.0 although I don't see any relevant changes.

The following program produces the output below it. I tried my best to make it as simple but complete as possible. It defines two exceptions, one of which does not inherit from `std::exception`, along with three matchers: one old matcher for each type and new generic matcher for both.

```C++
#include
#include
#include
#include
#include
#include

#include
#include

class Error : public std::runtime_error
{
public:
Error(std::string what, std::string extra)
: std::runtime_error(what)
, extra(extra)
{}

std::string toString() const
{
return std::string(what()) + ": " + extra;
}

private:
std::string extra;
};

CATCH_TRANSLATE_EXCEPTION(Error const& ex)
{
return ex.toString();
}

class CustomError
{
public:
CustomError(std::string msg)
: message(msg)
{}

std::string toString() const
{
return std::string("I don't inherit: ") + message;
}

private:
std::string message;
};

CATCH_TRANSLATE_EXCEPTION(CustomError const& ex)
{
return ex.toString();
}

using namespace Catch::Matchers;

struct OldErrorMatcher : MatcherBase
{
OldErrorMatcher(std::string m)
: m(m)
{}

bool match(const Error &ex) const override
{
auto s = ex.toString();
auto pos = s.find(": ");
return s.substr(0, pos) == m;
}

std::string describe() const override
{
return "describe()";
}

std::string m;
};

struct OldCustomErrorMatcher : MatcherBase
{
OldCustomErrorMatcher(std::string m)
: m(m)
{}

bool match(const CustomError &ex) const override
{
auto s = ex.toString();
auto pos = s.find(": ");
return s.substr(0, pos) == m;
}

std::string describe() const override
{
return "describe()";
}

std::string m;
};

struct NewErrorMatcher : MatcherGenericBase
{
NewErrorMatcher(std::string m)
: m(m)
{}

template
bool match(const Ex &ex) const
{
auto s = ex.toString();
auto pos = s.find(": ");
return s.substr(0, pos) == m;
}

std::string describe() const override
{
return "describe()";
}

std::string m;
};

void throwError()
{
throw Error("Oh no", "Bad luck");
}

void throwCustomError()
{
throw CustomError("Oh no");
}

TEST_CASE("Error") {
CHECK_THROWS_WITH(throwError(), StartsWith("Ahh yes"));
CHECK_THROWS_MATCHES(throwError(), Error, MessageMatches(StartsWith("Ahh yes")));
CHECK_THROWS_MATCHES(throwError(), Error, OldErrorMatcher("Ahh yes"));
CHECK_THROWS_MATCHES(throwError(), Error, NewErrorMatcher("Ahh yes"));
}

TEST_CASE("CustomError") {
CHECK_THROWS_WITH(throwCustomError(), StartsWith("Ahh yes"));
CHECK_THROWS_MATCHES(throwCustomError(), CustomError, OldCustomErrorMatcher("Ahh yes"));
CHECK_THROWS_MATCHES(throwCustomError(), CustomError, NewErrorMatcher("Ahh yes"));
}
```

The following is the output of the above program. Note the double space when using `MessageMatcher` (bug?).
```
Randomness seeded to: 1082738197

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
tests is a Catch2 v3.6.0 host application.
Run with -? for options

-------------------------------------------------------------------------------
Error
-------------------------------------------------------------------------------
/tmp/catch2experiment/test.cpp:128
...............................................................................

/tmp/catch2experiment/test.cpp:129: FAILED:
CHECK_THROWS_WITH( throwError(), StartsWith("Ahh yes") )
with expansion:
"Oh no: Bad luck" starts with: "Ahh yes"

/tmp/catch2experiment/test.cpp:130: FAILED:
CHECK_THROWS_MATCHES( throwError(), Error, MessageMatches(StartsWith("Ahh yes")) )
with expansion:
Oh no matches "starts with: "Ahh yes""

/tmp/catch2experiment/test.cpp:131: FAILED:
CHECK_THROWS_MATCHES( throwError(), Error, OldErrorMatcher("Ahh yes") )
with expansion:
Oh no describe()

/tmp/catch2experiment/test.cpp:132: FAILED:
CHECK_THROWS_MATCHES( throwError(), Error, NewErrorMatcher("Ahh yes") )
with expansion:
Oh no describe()

-------------------------------------------------------------------------------
CustomError
-------------------------------------------------------------------------------
/tmp/catch2experiment/test.cpp:135
...............................................................................

/tmp/catch2experiment/test.cpp:136: FAILED:
CHECK_THROWS_WITH( throwCustomError(), StartsWith("Ahh yes") )
with expansion:
"I don't inherit: Oh no" starts with: "Ahh yes"

/tmp/catch2experiment/test.cpp:137: FAILED:
CHECK_THROWS_MATCHES( throwCustomError(), CustomError, OldCustomErrorMatcher("Ahh yes") )
with expansion:
{?} describe()

/tmp/catch2experiment/test.cpp:138: FAILED:
CHECK_THROWS_MATCHES( throwCustomError(), CustomError, NewErrorMatcher("Ahh yes") )
with expansion:
{?} describe()

===============================================================================
test cases: 2 | 2 failed
assertions: 7 | 7 failed
```

Contributor guide

Open the contributing guide

Research direction

Start by reading the mentioned catch_translate_exception.hpp, catch_matchers_exception.hpp, and catch_matchers_templated.hpp headers, then trace how CHECK_THROWS_MATCHES formats failures. Compare that path with CHECK_THROWS_WITH and CATCH_TRANSLATE_EXCEPTION using the supplied reproducer. Done means translated exception messages and matcher descriptions are presented consistently, including the reported extra spacing.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.