garris / garris/BackstopJS

Unexpected differences between reference and test

Open
#1,140 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
7.2k
Forks
621
PR merge metrics
No merged PRs in 30d

Description

Hi, I run `backstop reference` then `backstop test` and I always get differences between both without making any changes.
Indeed, I got different differences everytime I run those 2 commands.
Do you see any sense on it?

Here you can find 2 different runs with just some minutes in between:
First: http://i.imgur.com/jan4DUX.png
Second: http://i.imgur.com/E2DJHJ1.png

This is my backstop.json:
`{
"id": "Design System",
"viewports": [
{
"label": "phone",
"width": 320,
"height": 480
},
{
"label": "tablet",
"width": 1024,
"height": 768
},
{
"label": "desktop",
"width": 1920,
"height": 1080
}
],
"onBeforeScript": "puppet/onBefore.js",
"onReadyScript": "puppet/onReady.js",
"scenarios": [
{
"label": "Home",
"url": "http://localhost:3000",
"referenceUrl": "",
"readyEvent": "",
"readySelector": "",
"delay": 2000,
"hideSelectors": [],
"removeSelectors": [],
"hoverSelector": "",
"clickSelector": "",
"postInteractionWait": 0,
"selectors": [],
"selectorExpansion": true,
"expect": 0,
"misMatchThreshold" : 0.1,
"requireSameDimensions": true
},
{
"label": "Home - SideNav",
"url": "http://localhost:3000",
"referenceUrl": "",
"readyEvent": "",
"readySelector": "",
"delay": 2000,
"hideSelectors": [],
"removeSelectors": [],
"hoverSelector": "",
"clickSelector": ".sidebar__hamburger",
"postInteractionWait": 0,
"selectors": [],
"selectorExpansion": true,
"expect": 0,
"misMatchThreshold" : 0.1,
"requireSameDimensions": true
}
],
"paths": {
"bitmaps_reference": "backstop_data/bitmaps_reference",
"bitmaps_test": "backstop_data/bitmaps_test",
"engine_scripts": "backstop_data/engine_scripts",
"html_report": "backstop_data/html_report",
"ci_report": "backstop_data/ci_report"
},
"report": ["browser"],
"engine": "puppeteer",
"engineOptions": {
"args": ["--no-sandbox"]
},
"asyncCaptureLimit": 5,
"asyncCompareLimit": 50,
"debug": false,
"debugWindow": false
}
`

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.