github / github/gh-gei

Add retries for the integration test asserts

Open
#909 3 comments 0 reactions 0 assignees View on GitHub
task
Dominant language
C#
Stars
478
Forks
146
Avg merge
7d 21h
Merged PRs (30d)
2

Description

We've hit a situation where we try to assert the final repo state, but github is not quite done digesting the data yet and returning a 404 when checking the commits (but when I manually checked out the repo 20 mins later it looked fine). This is causing the test assert to fail because it thinks the repo is uninitialized (which it probably is at that moment in time). But if the test just waited a few seconds it would probably sort itself out.

We could make our tests more robust by just wrapping the whole block of code that does the asserts into our retry library. If any asserts fail, just redo all the asserts (with the exponential backoff we have in our retry code).

Here's an example of a build failing because of this: https://github.com/github/gh-gei/actions/runs/4513628632/jobs/7948789505?pr=908#step:12:328

![image](https://user-images.githubusercontent.com/1508559/227607159-620aae2d-697d-4cc2-8325-39d88530dd00.png)
![image](https://user-images.githubusercontent.com/1508559/227607202-99ba2b5f-62e2-4913-baaf-0cdd49f9e28c.png)
![image](https://user-images.githubusercontent.com/1508559/227607225-d4344834-8dcd-46c2-9f3c-c50a9f20c6ec.png)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.