Azure / Azure/umock-c

umock either has some look-ahead or it doesn't have or both at the same time

Open
#135 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
36
Forks
27
Avg merge
32m
Merged PRs (30d)
3

Description

The following sort of self-contained code has admirable behavior at runtime:

```c
#define ENABLE_MOCKS
#include "umock_c/umock_c_prod.h"
MOCKABLE_FUNCTION(, void, sum, int, a, int, b);
MOCKABLE_FUNCTION(, void, whatIsDouble, int, a, int*, b);
#undef ENABLE_MOCKS

static void code_under_test1(void)
{
sum(2, 4);
sum(3, 5);
}

static void code_under_test2(void)
{
int nothing;
whatIsDouble(2, ¬hing);
whatIsDouble(3, ¬hing);
}

TEST_FUNCTION(umock_test_code_under_test1)
{
umock_c_reset_all_calls();

STRICT_EXPECTED_CALL(sum(2, 4));
code_under_test1();
STRICT_EXPECTED_CALL(sum(3, 5));
ASSERT_ARE_EQUAL(char_ptr, umock_c_get_expected_calls(), umock_c_get_actual_calls());
}

TEST_FUNCTION(umock_test_code_under_test2)
{
umock_c_reset_all_calls();

STRICT_EXPECTED_CALL(whatIsDouble(2, IGNORED_PTR_ARG));
code_under_test2();
STRICT_EXPECTED_CALL(whatIsDouble(3, IGNORED_PTR_ARG));
ASSERT_ARE_EQUAL(char_ptr, umock_c_get_expected_calls(), umock_c_get_actual_calls());
}
```

Note the extreme similarities between the 2 tests, the only difference being the second test uses a pointer instead of the int used in the first test.

Both tests (`umock_test_code_under_test1` and `umock_test_code_under_test2`) are poorly written and have a `STRICT_EXPECTED_CALL` after the code under test.

This is the expected behavior for `umock_test_code_under_test1` : after `code_under_test1` is called, there's going to be an actual call (`sum(3, 5);`) that is "extra" (actual). After `STRICT_EXPECTED_CALL(sum(3, 5));` there is another "extra" (expected) call. They should not be matched because the expected call was not existing at the time when the actual call happened.

However, umock_c will happily match them and in the case of `umock_test_code_under_test1` the test will happily pass.

![image](https://user-images.githubusercontent.com/9283365/69072597-d0492400-09e0-11ea-88b1-99bae7e35608.png)

`umock_test_code_under_test2` produces the expected behavior (that is, the test will fail). However, even when the test fails (as expected) the message on the screen doesn't really help with understanding "why" it failed, because it looks very "should pass":
![image](https://user-images.githubusercontent.com/9283365/69072387-692b6f80-09e0-11ea-8b62-9a131d918bbd.png)
"well - if the second argument is ignored, how come the calls didn't match?".

Overall the expectation is that both tests should fail. And when they fail - the message on the screen should indicate better "why they failed".

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by running the self-contained reproducer containing umock_test_code_under_test1 and umock_test_code_under_test2, focusing on STRICT_EXPECTED_CALL, umock_c_get_expected_calls(), and umock_c_get_actual_calls(). The change is done when both tests fail for the late expectations and the failure output explains why the calls did not match, including the ignored pointer argument case.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.