Snapshot of `file` to '...' has changed
Nobody has claimed this yet.
- Dominant language
- R
- Stars
- 123
- Forks
- 23
- PR merge metrics
- No merged PRs in 30d
Description
I created a simple snapshot test:
library(shinytest2)
test_that("{shinytest2} recording: startup", {
app <- AppDriver$new(variant = platform_variant(), name = "startup", seed = 3812,
height = 976, width = 1619)
app$expect_screenshot()
})
...which passes when I run shinytest2::test_app() (mac-OS 12.6). However, my GitHub action on macos-latest fails:
✔ | F W S OK | Context
date, intersect, setdiff, union
⠏ | 0 | shinytest2
⠋ | 1 0 | shinytest2
✖ | 1 0 | shinytest2 [11.0s]
────────────────────────────────────────────────────────────────────────────────
Failure (test-shinytest2.R:7): {shinytest2} recording: startup
Snapshot of `file` to 'shinytest2/startup-001.png' has changed
Error: Error: Test failures
Run `testthat::snapshot_review('shinytest2/')` to review changes
Execution halted
Backtrace:
1. app$expect_screenshot()
at test-shinytest2.R:7:2
2. shinytest2:::app_expect_screenshot_and_variant(...)
3. shinytest2:::app_expect_screenshot(self, private, ...)
4. shinytest2:::app__expect_snapshot_file(...)
6. testthat::expect_snapshot_file(...)
My check-app.yaml file:
on:
push:
branches: [main, master]
pull_request:
branches: [main, master]
name: my_shiny_app
jobs:
test-app:
runs-on: ${{ matrix.os }}
name: ${{ matrix.os }} (${{ matrix.r }})
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest]
r: [4.2.1]
env:
GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
R_KEEP_PKG_SOURCE: yes
steps:
- uses: actions/checkout@v2
- uses: r-lib/actions/setup-pandoc@v2
- uses: r-lib/actions/setup-r@v2
with:
r-version: ${{ matrix.r }}
http-user-agent: ${{ matrix.config.http-user-agent }}
use-public-rspm: true
- uses: r-lib/actions/setup-renv@v2
- uses: rstudio/shinytest2/actions/test-app@v1
with:
path: "."
The CI test OS: macOS 12.6.1
If I had to guess on what the problem is, I would bet on the png width slightly changing. It appears that the png width doesn't stay constant in all conditions, even though width= is set via AppDriver$new()
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The reproduction is in test-shinytest2.R at line 7, using AppDriver$new() and app$expect_screenshot(). Run shinytest2::test_app() locally and the rstudio/shinytest2/actions/test-app@v1 workflow on macos-latest, then compare shinytest2/startup-001.png and use testthat::snapshot_review('shinytest2/'). Done means identifying the cause of the platform-specific snapshot difference and establishing a reliable test result.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, r
- Domain
- ci-cd, operating-systems, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100