Inconsistent test results using v5.3.0 or v.5.0.7 with no docker (similar to #1303)
- Dominant language
- JavaScript
- Stars
- 7.2k
- Forks
- 621
- PR merge metrics
- No merged PRs in 30d
Description
When I compare some url with itself I have some random fail/pass results.
Here my config file:
````
module.exports = {
id: "uniq_id_config",
viewports: [
{
label: "tablet",
width: 1024,
height: 768
}
],
onBeforeScript: "puppet/onBefore.js",
onReadyScript: "puppet/onReady.js",
scenarios: [
{
label: 'Home',
misMatchThreshold : 0.5,
referenceUrl: 'https://dewalt.com/',
url: 'https://dewalt.com/'
}
],
paths: {
bitmaps_reference: "backstop_data/bitmaps_reference",
bitmaps_test: "backstop_data/bitmaps_test",
engine_scripts: "backstop_data/engine_scripts",
html_report: "backstop_data/html_report",
ci_report: "backstop_data/ci_report"
},
report: ["browser"],
engine: "puppeteer",
engineOptions: {
args: ["--no-sandbox"]
},
asyncCaptureLimit: 5,
asyncCompareLimit: 50,
debug: false,
debugWindow: false
}
````
I'm using 'https://dewalt.com/' to test. I've tested with other url like: 'https://itc.ua/' - it have the same behavior.
Also tested with the last version of backstopjs - it have the same behavior.
link to repo:
https://github.com/akmil/backstop-demo
Contributor guide
Assessment
This issue has not been assessed yet.