Evaluating tests in their own frame to match test that behavior
- Dominant language
- R
- Stars
- 145
- Forks
- 16
- PR merge metrics
- No merged PRs in 30d
Description
Should patrick tests not be running in their own call stack, similar to testthat?
For various reasons I like to use defer to do cleanup after a test, for example in mocking or setting global options.
However, with Patrick it seems like the test environment is run in the global environment/call frame, so the mocks do not get cleaned up when the test is finished running (because the environment they were mocked in never exits).
Ideally of course I wouldn't have to modify the global environment in my tests, but for the time being I must, and testthat behaves as expected.
This was a particularly tricky issue to discover.
This is shown in the below reproduction, the value of `val` is restored after testthat calls, because although the environment is still the global environment, the frame number has changed (and presumably that frame ends, triggering a defer call). However, with Patrick, `val` is not cleaned up after the mock test, resulting in the second test failing because the global value has changed.
If this is intended behavior, I would appreciate a suggestion for a work around.
``` r
{
mock_value <- function(name, replacement, parent = 1) {
old_value <- get(name, envir = .GlobalEnv)
assign(name, replacement, envir = .GlobalEnv)
withr::defer({
assign(name, old_value, envir = .GlobalEnv)
print("ran defer block")
}, envir = parent.frame(parent))
}
val = TRUE
testthat::test_that("", {
mock_value("val", FALSE)
print("initial test that mock")
print(val)
print(sys.frame())
print(sys.nframe())
testthat::expect_false(val)
})
print(val)
testthat::test_that("", {
print("second test that no mock")
print(val)
print(sys.frame())
print(sys.nframe())
testthat::expect_true(val)
})
print(val)
patrick::with_parameters_test_that(":", .cases = list(a = ""),
{
mock_value("val", FALSE)
print("initial with parameters test that mock")
print(val)
print(sys.frame())
print(sys.nframe())
testthat::expect_false(val)
})
print(val)
patrick::with_parameters_test_that(":", .cases = list(a = ""),
{
print("second with parameters test that no mock")
print(val)
print(sys.frame())
print(sys.nframe())
testthat::expect_true(val)
})
print(val)
}
#> [1] "initial test that mock"
#> [1] FALSE
#>
#> [1] 52
#> [1] "ran defer block"
#> Test passed with 1 success π.
#> [1] TRUE
#> [1] "second test that no mock"
#> [1] TRUE
#>
#> [1] 52
#> Test passed with 1 success π.
#> [1] TRUE
#> [1] "initial with parameters test that mock"
#> [1] FALSE
#>
#> [1] 0
#> Test passed with 1 success π₯.
#> [1] FALSE
#> [1] "second with parameters test that no mock"
#> [1] FALSE
#>
#> [1] 0
#> ββ Failure: : a= βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
#> Expected `val` to be TRUE.
#> Differences:
#> `actual`: FALSE
#> `expected`: TRUE
#>
#> Backtrace:
#> β
#> 1. ββrlang::eval_tidy(code, test_args)
#> 2. ββtestthat::expect_true(val)
#> Error in `pmap()`:
#> βΉ In index: 1.
#> Caused by error:
#> ! Test failed with 1 failure and 0 successes.
```
Created on 2026-05-08 with [reprex v2.1.1](https://reprex.tidyverse.org)
Contributor guide
Assessment
This issue has not been assessed yet.