MarketSquare / MarketSquare/robotframework-retryfailed
Add an option to retry based on error message
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 19
- Forks
- 8
- PR merge metrics
- No merged PRs in 30d
Description
# Requirement
## Short Story
I have flaky test cases that have 60% chance to fail, but only in test automation, not in manual testing. Retrying helps. All test cases fail with almost the same error message. I would like to rerun those test cases that fail with an expected error message. All other errors are actual failures and I do **not** want to retry them.
## Long Story
I have browser test cases running against a SUT implemented in MaterialUI. MaterialUI in itself seems to be flaky when being used too fast (even in manual testing). However, I can't change the SUT, so test cases have to work on MaterialUI, too. As the error messages all hold similar cause (only details in the stacktrace differ), i would like to retry these flaky test cases automatically. All other errors shall count as failure without a retry.
# Suggestion
```robotframework
*** Test Cases ***
Test Super Flaky MaterialUI
[Tags] test:retry(3)(MaterialPopover) # flaky error message always contains "MaterialPopover"
Login User
....
```
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Robot Framework listener and tag-based retry handling described in the issue, then trace how retry counts and failure messages are currently evaluated. Define completion as supporting a tag such as test:retry(3)(MaterialPopover), retrying only failures whose error message contains the requested text while preserving normal failures for other errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa, tooling
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100