cucumber / cucumber/cucumber-cpp

Steps throwing exceptions not reported as an error (Boost.Test)

Open
#222 2 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
C++
Stars
330
Forks
140
PR merge metrics
No merged PRs in 30d

Description

## Summary

We encountered the issue that step definitions which throw an exception are reported as passed, when using the Boost.Test framework and cucumber-cpp.

## Expected Behavior

Our expectation was that a step which throws an exception (for whatever reason) should be reported as a failed step by cucumber-cpp.

## Current Behavior

The step is reported as passed. The rest of the step definition after the exception gets thrown, doesn't get executed.

## Possible Solution

Not familiar with Boost.Test nor cucumber-cpp source code, but while debugging this, I noticed the following:
The exception is caught by `unit_test_monitor_t::execute_and_translate(...)` in unit_test_monitor.ipp and gets reported to test observers.
However, the BoostDriver only considers the test log for the decision to rate a step as success or failure, but exceptions don't show up in this log for some reason.
We could successfully trigger a failed step by adding something along the following lines after `runWithMasterSuite()` (needs `#include `):
~~~~
auto results = ::boost::unit_test::results_collector.results(testCase->p_id);
if (!results.passed())
{
std::string description = "Test aborted/failed with result code " + std::to_string(results.result_code());
return InvokeResult::failure(description.c_str());
}
~~~~

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.