canonical / canonical/checkbox
Change metabox "AssertPrinted" to work line by line
- Dominant language
- Python
- Stars
- 52
- Forks
- 81
- Avg merge
- 2d 17h
- Merged PRs (30d)
- 69
Description
### Enhancement Proposal
The default behavior of “AssertPrinted” in metabox is to look for a specific string in the whole output, instead of line by line. This is a weird behavior to have by default, because If you have several “AssertPrinted" in a row, the order or the number of times it's repeated won't matter.
E.g.
```
steps = [
Start(),
AssertPrinted("template_validation_invalid_fields_somename"),
AssertPrinted("Outcome: job passed"),
AssertPrinted("template_validation_invalid_fields_invalid_body"),
AssertPrinted("Outcome: job failed"),
AssertPrinted("template_validation_missing_parameter_MISSING_PARAM_1"),
AssertPrinted("Outcome: job failed"),
AssertRetCode(1),
]
```
This test won't check if the output strings have been printed in the order it's declared, but if the string is in the whole output. That makes it hard to check the outcome of a specific job.
We should make it so "AssertPrinted" checks for the output line by line, so the order in which the strings are printed is maintained.
Contributor guide
Research direction
Start by locating the AssertPrinted implementation and its existing tests in the Checkbox repository. Read how output is currently matched, then run the relevant tests and verify that consecutive assertions preserve output order and repeated lines are checked correctly.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100