dmlc / dmlc/xgboost

[R] 'MultiClassEvaluation: label must be in [0, num_class), num_class=16 but found 16 in label' by using grid search in caret

Open
#6,462 0 comments 0 reactions 0 assignees View on GitHub
status: help wanted
Dominant language
C++
Stars
28.8k
Forks
8.9k
Avg merge
1d 12h
Merged PRs (30d)
54

Description

I know for multi-classification problem, `objective="multi:softmax"`, you have to transfer the target into classes starting from 0. For example, if 16 classes, target should be 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15. I can run a multiclass xgboost model without any issue.

However, when I was trying grid search to do the parameter tuning, it asked me to transfer the target into factor levels as required by `metric = "Accuracy"` for train function in caret (https://www.rdocumentation.org/packages/caret/versions/4.47/topics/train).
These two are conflicted and I was not able to run either of them.

If I set the target to 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15 (numeric), I was not able to run the grid search and got error message -
`Error: Metric Accuracy not applicable for regression models`

If I set the target to 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15 but change them to factors, I still couldn't run the grid search and got error message -
`Error: At least one of the class levels is not a valid R variable name; This will cause errors when class probabilities are generated because the variables names will be converted to X0, X1, X2, X3, X4, X5, X6, X7, X8, X9, X10, X11, X12, X13, X14, X15 . Please use factor levels that can be used as valid R variable names (see ?make.names for help).`

If I set the target to g0, g1, g2, g3, g4, g5, g6, g7, g8, g9, g10, g11, g12, g13, g14, g15, it will start running the grid search but I will get the error message in the stack trace -
```
[20:38:51] amalgamation/../src/metric/multiclass_metric.cu:32: Check failed: label_error >= 0 && label_error < static_cast(n_class): MultiClassEvaluation: label must be in [0, num_class), num_class=16 but found 16 in label
Stack trace:
[bt] (0) 1 xgboost.dylib 0x000000012c0c21d9 dmlc::LogMessageFatal::~LogMessageFatal() + 57
[bt] (1) 2 xgboost.dylib 0x000000012c11b604 xgboost::metric::MultiClassMetricsReduction::CheckLabelError(int, unsigned long) const + 228
[bt] (2) 3 xgboost.dylib 0x000000012c11b4fe xgboost::metric::MultiClassMetricsReduction::CpuReduceMetrics(xgboost::HostDeviceVector const&, xgboost::HostDeviceVector const&, xgboost::HostDeviceVector const&, unsigned long) const + 398
[bt] (3) 4 xgboost.dylib 0x000000012c11aefa xgboost::metric::EvalMClassBase

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.