marbles can't execute assertions that don't accept a msg argument
- Dominant language
- Python
- Stars
- 155
- Forks
- 22
- PR merge metrics
- No merged PRs in 30d
Description
* marbles.core version: 0.9.4
* marbles.mixins version: 0.9.4
* Python version: 3.6.2
* Operating System: Linux
### Description
The [unittest docs](https://docs.python.org/3/library/unittest.html#test-cases) specify that:
> All the assert methods accept a msg argument that, if specified, is used as the error message on failure (see also `longMessage`). Note that the _msg_ keyword argument can be passed to `assertRaises()`, `assertRaisesRegex()`, `assertWarns()`, `assertWarnsRegex()` only when they are used as a context manager.
If these assertion methods are used as functions and not context managers, marbles will try it's best to find the _msg_ argument, but because these methods don't accept a _msg_ argument when they're used as functions, marbles will end up grabbing an argument that's intended for the callable and then the callable will be sad.
### What I Did
I originally found this while trying to run unittest's own tests (specifically, the [test_assertions.py](https://github.com/python/cpython/blob/3.6/Lib/unittest/test/test_assertions.py) tests) under marbles, but you can reproduce this specific bug with the following test case:
```python
import unittest
class MyTestCase(unittest.TestCase):
def test_foo(self):
self.assertRaises(self.failureException,
self.assertAlmostEqual, 1.0000001, 1.0)
if __name__ == '__main__':
unittest.main()
```
running this produces the following:
```
$ python -m marbles test.py
E
======================================================================
ERROR: test_foo (test.MyTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "test.py", line 8, in test_foo
self.assertAlmostEqual, 1.0000001, 1.0)
File ".../lib/python3.6/site-packages/marbles/core/marbles.py", line 538, in wrapper
return attr(*args, msg=annotation, **kwargs)
File ".../lib/python3.6/unittest/case.py", line 733, in assertRaises
return context.handle('assertRaises', args, kwargs)
File ".../lib/python3.6/unittest/case.py", line 178, in handle
callable_obj(*args, **kwargs)
File ".../lib/python3.6/site-packages/marbles/core/marbles.py", line 538, in wrapper
return attr(*args, msg=annotation, **kwargs)
TypeError: assertAlmostEqual() missing 1 required positional argument: 'second'
----------------------------------------------------------------------
Ran 1 test in 0.002s
FAILED (errors=1)
```
### Additional context
This is related to a more general issue which is that marbles expects every assertion to accept a _msg_ argument. If a non-unittest assertion (e.g., mixin assertion, a user-defined assertion, etc.) doesn't accept a _msg_ argument, marbles won't be able to execute that assertion.
An example of this occurs in unittest's own tests. The `TestLongMessage` tests define a custom assertion, [`assertMessages`](https://github.com/python/cpython/blob/c4750959acbfc3057f12aaec832483ba30898d1c/Lib/unittest/test/test_assertions.py#L182) that doesn't accept a _msg_ argument. When a test method calls this assertion, marbles will dig into its arguments to try to pull out the _note_ and _msg_ arguments, and then try to make the assertion. In this case, one or two things could happen:
1. marbles might grab something that is not a _msg_
1. when marbles tries to make the assertion it will pass what it thought was the _msg_ as a keyword argument to the original assertion, causing the original assertion to complain about getting an unexpected keyword argument
Contributor guide
Research direction
Start at marbles/core/marbles.py around the wrapper reported at line 538, then reproduce the failure with the provided unittest case. Compare the wrapper's handling of msg with unittest's assertRaises and the assertion methods in test_assertions.py. Done means assertions that do not accept msg, including function-form assertRaises calls and custom assertions, execute without receiving an unintended argument.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100