MarketSquare / MarketSquare/robotframework-retryfailed

Add an option to retry based on error message

Open
#12 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
19
Forks
8
PR merge metrics
No merged PRs in 30d

Description

# Requirement

## Short Story
I have flaky test cases that have 60% chance to fail, but only in test automation, not in manual testing. Retrying helps. All test cases fail with almost the same error message. I would like to rerun those test cases that fail with an expected error message. All other errors are actual failures and I do **not** want to retry them.

## Long Story
I have browser test cases running against a SUT implemented in MaterialUI. MaterialUI in itself seems to be flaky when being used too fast (even in manual testing). However, I can't change the SUT, so test cases have to work on MaterialUI, too. As the error messages all hold similar cause (only details in the stacktrace differ), i would like to retry these flaky test cases automatically. All other errors shall count as failure without a retry.

# Suggestion

```robotframework
*** Test Cases ***
Test Super Flaky MaterialUI
[Tags] test:retry(3)(MaterialPopover) # flaky error message always contains "MaterialPopover"
Login User
....
```

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Robot Framework listener and tag-based retry handling described in the issue, then trace how retry counts and failure messages are currently evaluated. Define completion as supporting a tag such as test:retry(3)(MaterialPopover), retrying only failures whose error message contains the requested text while preserving normal failures for other errors.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa, tooling
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.