Unseen Categories Using Categorical Re-coder
- Dominant language
- C++
- Stars
- 28.8k
- Forks
- 8.9k
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 54
Description
I’m using the latest version (3.1.1) and testing the new Categorical Re-coder feature in Python. During prediction, I encountered the following error:
Found a category not in the training set.
In previous versions, I handled this by using pd.Categorical and manually storing the categories during training. At inference time, any unseen category was automatically converted to NaN, which allowed the model to process it without failing.
With the new Categorical Re-coder in 3.1, if an unseen category appears during inference, the process fails with the above error.
Is this the intended behavior for the new Categorical Re-coder?
If so, what is the recommended approach for handling unseen categories during inference?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the prediction failure with XGBoost 3.1.1, the new Categorical Re-coder, and an inference category absent from training. Compare this with the earlier pd.Categorical workflow described in the issue. Done means establishing whether unseen categories should be accepted and documenting or implementing the recommended inference behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100