garris / garris/BackstopJS

Backstop test fails with "ENOENT: no such file or directory" if a new scenario is added

Open
#403 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
JavaScript
Stars
7.2k
Forks
621
PR merge metrics
No merged PRs in 30d

Description

Hi there!

If a new scenario is added our backstop test builds are failing in the comparison phase because backstop can't find the image for the new scenario inside the reference image folder.

The solution for the failing build is to create new reference images on that branch. This can lead to an erroneous state of the reference images on that branch if undesired changes have been made to the styling of other views.

My proposal would be to check if the reference image for a scenario is existing and marking the scenario as failed if it's not. Another solution would be to add a parameter to configure that behavior explicitly like "continue on failure". That way the failed diffs would still be marked for the existing scenarios.

Example output of the Jenkins build:
```
compare | OK: Oldpage_document_desktop.png
events.js:160
throw er; // Unhandled 'error' event
^

Error: ENOENT: no such file or directory, open '/src/src/test/backstop/data/bitmaps_reference/NewPage_document_desktop.png'
at Error (native)
```

Any thoughts or ideas on that?

Thanks for your time and effort,
Robert

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.