Make ztest more Zeek-like
- Dominant language
- Zeek
- Stars
- 2
- Forks
- 5
- PR merge metrics
- No merged PRs in 30d
Description
This is a collection of issues we should consider fixing before integrating Ztest into Zeek, see zeek/zeek#1475.
- [ ] Fix structure to more in line what one would expect from a package
- [ ] Remove Ruby dependency of test suite, porting to BTest seems reasonable
- [ ] Make output stable, likely needed for BTest
- [ ] Tests are currently stored in a `table` `tests` which has no stable
iteration order. Consider introducing a `vector` to iterate over and a
lookup table from test name to index, or any other stable dictionary type
available in standard Zeek script
- [ ] Test functions are stored as lambdas which produce unstable names in
default-enabled backtraces. Consider disabling backtraces by default as
they require postprocessing.
- [ ] Argument stringification in assertion macros can produce unstable
results for e.g., macros like `assert_equal` or sim. ZTest currently uses
`cat` to compute a representation for printing. For e.g., `table` or `set`
args this can produce unstable output. Consider introducing a helper macro
which ensures a consistent representation. We might be able to reuse what
we develop here to make uses of `tests` stable.
- [ ] Consider making assertion macros standalone. ZTest assertion macros
(e.g., `assert`, `assert_equal`, or sim.) currently assume that they are used
from a `ZTest::test` inside a `ZTest::test_suite` so they cannot be used
standalone. OTOH, having them available from tests not currently written in
the full `ZTest` framework would be useful, e.g., to slowly phase in ZTest
use in existing tests. When used standalone these macros could on failure
e.g., produce an error message and should probably call `exit` with an error
code.
- [ ] Consider aborting tests on assertion failures. It is currently impossible to check preconditions with assertion macros as they do not stop the remainder of the test. This can lead to cascading failures where the first failure might have been sufficient to detect that the test should have been aborted.
- [ ] Consider to make message string for assertion macros optional.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.