MarketSquare / MarketSquare/robotframework-retryfailed
Infinite loop of retries for RetryFailed listener
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 19
- Forks
- 8
- PR merge metrics
- No merged PRs in 30d
Description
I have a test run set up in GCP where in options I included RetryFailed listener and skips for tags "skip":
```
substitutions
...
_ROBOT_OPTIONS: --listener RetryFailed --exclude skip
_HEADLESS: 'True'
...
```
Inside the test case I use tag:
```
[Tags] test:retry(1)
```
I faced an issue that test run is stuck in an infinite loop. In the logs I can see messages:

...and again, and again: PASS skip FAIL skip. And it retries like this for hours until the job stops by timeout. I have a scheduled run.
I ran the same exact configuration tests run in Docker container locally and RetryFailed listener works as expected, without this infinite loop of retries.
I don't know if anyone stumbled upon this issue, but I found no similar information as to why it might happen.
I'm not sure, but I think the issue might happen because of the thing that in this block of code it sets up the result.status = "SKIP", sets the retry count back to 0, and on the next retry, which shouldn't happen, it loops back again:
```
def end_test(self, test, result):
if self.retries and self._original_log_level is not None:
BuiltIn()._context.output.set_log_level(self._original_log_level)
if not self.max_retries:
self.retries = 0
return
if result.status == "FAIL":
if self.retries < self.max_retries:
index = test.parent.tests.index(test)
test.parent.tests.insert(index + 1, test)
result.status = "SKIP"
result.message += "\nSkipped for Retry"
self.retried_tests.append(test.longname)
self.retries += 1
return
else:
result.message += (
f"{linebreak * bool(result.message)}[RETRY] FAIL on {self.retries}. retry."
)
else:
if self.retries:
result.message += (
f"{linebreak * bool(result.message)}[RETRY] PASS on {self.retries}. retry."
)
self.retries = 0
return
```
The expected behavior is that it should mark test as FAIL after defined number of retries, and then skip to the next test run
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the RetryFailed listener's end_test method shown in the report, then reproduce the _ROBOT_OPTIONS configuration with --listener RetryFailed --exclude skip in GCP and compare it with the local Docker run. Trace the PASS/skip/FAIL/skip sequence and verify that a test is marked FAIL after the configured retries and execution advances to the next test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100