tidymodels / tidymodels/workflows
warning for novel levels appear even if you use `step_novel()`
Nobody has claimed this yet.
- Dominant language
- R
- Stars
- 211
- Forks
- 26
- Avg merge
- 1h 58m
- Merged PRs (30d)
- 1
Description
If you use step_novel() in a workflow, you can get the warning Warning: Novel levels found in column 'x': '3'. The levels have been removed, and values have been coerced to 'NA'., which seems like a mistake since step_novel() should be able to take care of it.
I think the warning comes from hardhat::scream().
library(tidymodels)
data <-
data.frame(
y = as.factor(c(0, 1, 0, 0, 0)),
x = as.factor(c(0, 1, 2, 0, 0))
)
test <-
data.frame(
x = as.factor(c(0, 1, 2 ,3))
)
model_spec <- decision_tree() %>%
set_engine("rpart") %>%
set_mode("classification")
rec_spec <-
recipe(y ~ x, data = data) %>%
step_novel(x)
rf_workflow <-
workflow() %>%
add_recipe(rec_spec) %>%
add_model(model_spec)
final_model <- fit(rf_workflow, data)
test_predict <- predict(final_model, test)
#> Warning: Novel levels found in column 'x': '3'. The levels have been removed,
#> and values have been coerced to 'NA'.
test_predict
#> # A tibble: 4 x 1
#> .pred_class
#> <fct>
#> 1 0
#> 2 0
#> 3 0
#> 4 0
Created on 2021-07-04 by the reprex package (v2.0.0)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the warning with the workflow and recipe shown in the issue, then trace the prediction path through hardhat::scream(). Confirm how step_novel() handles the unseen level and whether the warning is still appropriate; done means the example no longer emits a misleading warning while prediction remains unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- r
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 40/100