garris / garris/BackstopJS

Test failing in Smoke-test, fonts not loaded?

Open
#1,356 8 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
7.2k
Forks
621
PR merge metrics
No merged PRs in 30d

Description

I have this situation that some tests are sometimes randomly failing.
I noticed the same situation in the smoke-tests.

I cannot figure out why this test is constantly failing, I've tried different setups, but I don't understand why.

The code is in this branch:
https://github.com/klodoma/BackstopJS/tree/debug/failing-tests

You can download the test results and artifacts here:
https://gitlab.com/klodoma/BackstopJS/-/pipelines/353734894

I want to mention that I have this situation quite often in my tests, that for some reason the "rendered text size" is rendered differently. Maybe it's a bug in pupeteer and it's quite common.

One suspect is that the screenshot is made before the font-files are loaded.

Any thoughts are welcomed.

![image](https://user-images.githubusercontent.com/944350/129534800-501397ec-0c80-44f4-a344-e92c83571714.png)

```js
{
label: 'keyPressSelector',
url: `${URL}/examples/featureTests/index.html`,
keyPressSelectors: [
{
selector: 'input[placeholder="Email"]',
keyPress: 'marcdacz@backstopjs.com'
},
{
selector: 'input[placeholder="Password"]',
keyPress: '1234'
}
],
selectors: ['div[id=navbar]'],
postInteractionWait: 2000,
viewports: [
{
label: 'Desktop',
width: 800,
height: 300
}
]
}
```

Contributor guide

Open the contributing guide

Research direction

Start by checking the debug/failing-tests branch and reproducing the smoke-test case using the shown keyPressSelector configuration for examples/featureTests/index.html. Compare the downloaded GitLab pipeline artifacts and inspect whether font loading or screenshot timing differs between runs. Done means the intermittent failure has a confirmed cause and a reliable test result.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.