canonical / canonical/checkbox

Change metabox "AssertPrinted" to work line by line

Open
#1,966 2 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
52
Forks
81
Avg merge
2d 17h
Merged PRs (30d)
69

Description

### Enhancement Proposal

The default behavior of “AssertPrinted” in metabox is to look for a specific string in the whole output, instead of line by line. This is a weird behavior to have by default, because If you have several “AssertPrinted" in a row, the order or the number of times it's repeated won't matter.
E.g.
```
steps = [
Start(),
AssertPrinted("template_validation_invalid_fields_somename"),
AssertPrinted("Outcome: job passed"),
AssertPrinted("template_validation_invalid_fields_invalid_body"),
AssertPrinted("Outcome: job failed"),
AssertPrinted("template_validation_missing_parameter_MISSING_PARAM_1"),
AssertPrinted("Outcome: job failed"),
AssertRetCode(1),
]
```
This test won't check if the output strings have been printed in the order it's declared, but if the string is in the whole output. That makes it hard to check the outcome of a specific job.

We should make it so "AssertPrinted" checks for the output line by line, so the order in which the strings are printed is maintained.

Contributor guide

Open the contributing guide

Research direction

Start by locating the AssertPrinted implementation and its existing tests in the Checkbox repository. Read how output is currently matched, then run the relevant tests and verify that consecutive assertions preserve output order and repeated lines are checked correctly.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.