nunit / nunit/docs

Improve Documentation To Highlight Reliance on Implementation Details of Assert As Bad

Open
#294 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area:framework
Dominant language
Dockerfile
Stars
600
Forks
158
Avg merge
16h 8m
Merged PRs (30d)
21

Description

This is a continuation of this discussion: #168 I completely agree with @CharliePoole on this, but I would suggest that the documentation be improved to highlight the situation.

-- Begin Documentation Improvement Suggestion --

Behavior of Asserts
[Test]
public void TestAssertCatch()
{
    try
    {
        Assert.That(true, Is.False);
    }
    catch
    {
    }
}

When looking at this example you may expect NUnit to Assert that true != false. However in versions of NUnit Prior to 3.6.0.0 this test would pass due to an implementation detail of Assert. In newer versions of NUnit this will Assert (and fail the test) as a design decision of the NUnit.org Team.

In cases where this is used, it was probably used in scenarios where better APIs could be leveraged.

Consider the following test which is trying to Assert that an Exception is thrown/not thrown:

public void TestException()
{
    try
    {
        GuessTheNumber(42);

        // This code only executes if no exception has occurred
        Assert.Fail("An expected exception did not occur");
    }
    catch(Exception ex)
    {
        Assert.That(ex.Message, Is.EqualTo("The Answer"), "An expected exception did not occur");
    }
}

public static void GuessTheNumber(int number)
{
    if(number == 42)
    {
        throw new Exception("The Answer");
    }
}

Instead this should be rewritten to use either Assert.That or Assert.Throws like so:

[Test]
public void TestExceptionAssertThat()
{
    Assert.That(() => GuessTheNumber(42), Throws.InstanceOf<Exception>().With.Property("Message").EqualTo("The Answer"), "An expected exception did not occur");
}

Or Assert.Throws Style

[Test]
public void TestExceptionAssertThrows()
{
    Exception ex = Assert.Throws<Exception>(() => GuessTheNumber(42), "An expected exception did not occur");
    Assert.That(ex.Message, Is.EqualTo("The Answer"), "An expected exception did not occur");
}

-- End Documentation Improvement --

As Charlie mentioned:

I have asked the submitter of the issue to give a motivating example that would help us meet the need

It would be nice to hear what other scenarios end users are encountering that cause them to need to capture the exception of the Assert. The above example is just what we found in our code base during our 2.6.4->3.11.0 Convert.

Thank you for this tool; it is invaluable to us.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading discussion #168 and the documentation improvement suggestion in this issue, then locate the relevant NUnit assertion documentation. Done means documenting the Assert behavior change, explaining why relying on implementation details is discouraged, and showing the Assert.That and Assert.Throws alternatives.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
documentation, testing-qa
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.