bazelbuild / bazelbuild/bazel-skylib
Prevent printing output to allow failure testing of custom asserts
- Dominant language
- Starlark
- Stars
- 444
- Forks
- 202
- PR merge metrics
- No merged PRs in 30d
Description
It's currently a bit tricky to do failure tests of custom assert functions. One particular wart is that any expected failure will call print() and output a bogus failure message, even though the test passes. e.g.
```
def assert_even(env, n):
asserts.true(env, n %2 == 0)
def test_custom_assert(ctx):
env = unittest.begin(ctx)
assert_even(env, 1)
env.failures.pop() # prevent failure from propagating
return unittest.end(ctx)
```
The above will pass, as expected, but print an bogus error. The logic for this is in `unittest.bzl#_fail`:
```
def _fail(env, msg):
print(full_msg)
env.failures.append(full_msg)
```
Here's a couple proposed solutions:
1. Get the "print" function from the env struct. This allow callers to mock it out with their own callable.
2. Add a "bool print_on_failure" optional attribute to the env struct. This would skip calling print() if true.
Contributor guide
Research direction
The relevant logic is in unittest.bzl#_fail; start by reading how custom assertions record failures and inspect the existing unittest tests. Done means a custom-assert failure can be verified through the recorded environment without producing the bogus output, with regression coverage for that behavior.
Written by the indexing model from the issue text.
Assessment
- Domain
- build-system, testing
- Issue type
- Feature
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100