Thinking about how we test parsnip model registration with extension packages

Open
#186 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Stale
Tech stack
r
Domain
testing-qa

Research direction

Start by reproducing the example with parsnip::get_model_env(), test_that(), and library(baguette), comparing results when tests run individually versus together. Investigate how the baguette .onload() registration persists across tests. Done means extension registration no longer makes results depend on test order, with coverage for the isolated behavior.

Written by the indexing model from the issue text.

Description

upkeep

This is related to #185. Which was darn hard to figure out.

What happens is that the .onload() of baguette will trigger in a test. Which is what we want. But it will stay triggered for any of the following tests. Such that we get different results sometimes depending on if we test all files or one file. Also, it violates some soft of test smell, since the order of tests matter

library(parsnip)
library(testthat)

parsnip::get_model_env()$bag_tree_args
#> # A tibble: 0 × 5
#> # ℹ 5 variables: engine <chr>, parsnip <chr>, original <chr>, func <list>,
#> #   has_submodel <lgl>

# could be in a previous file
test_that("load baguette", {
  library(baguette)
})
#> ── Skip: load baguette ─────────────────────────────────────────────────────────
#> Reason: empty test

parsnip::get_model_env()$bag_tree_args
#> # A tibble: 6 × 5
#>   engine parsnip         original func             has_submodel
#>   <chr>  <chr>           <chr>    <list>           <lgl>       
#> 1 rpart  class_cost      cost     <named list [2]> FALSE       
#> 2 rpart  tree_depth      maxdepth <named list [2]> FALSE       
#> 3 rpart  min_n           minsplit <named list [2]> FALSE       
#> 4 rpart  cost_complexity cp       <named list [2]> FALSE       
#> 5 C5.0   class_cost      cost     <named list [2]> FALSE       
#> 6 C5.0   min_n           minCases <named list [2]> FALSE
Dominant language
R
Stars
23
Forks
0
Avg merge
3h 57m
Merged PRs (30d)
1

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from tidymodels/extratests

All issues in tidymodels/extratests

Similar issues

More R issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.