dmlc / dmlc/xgboost

xgb.cv´s folds list of test indices and "unnatural" corresponding train indices

Open
#4,657 2 comments 0 reactions 0 assignees View on GitHub
cross-validation
Dominant language
C++
Stars
28.8k
Forks
8.9k
Avg merge
1d 12h
Merged PRs (30d)
54

Description

When one gives a list of test indices, the most natural selection of train indices are the remaining lines. But in the code, all other teste indices are taken as train indices:
```{r}
bst_folds <- lapply(seq_along(folds), function(k) {
dtest <- slice(dall, folds[[k]])
dtrain <- slice(dall, unlist(folds[-k]))
handle <- xgb.Booster.handle(params, list(dtrain, dtest))
list(dtrain = dtrain, bst = handle, watchlist = list(train = dtrain,
test = dtest), index = folds[[k]])
})
})
```
This makes the selection of test/train very difficult. For example, one wants to take different tails sizes as test indices:
```{r}
n=nrow(data) # =2275
ts1=seq(1000,125,-125) #~8 to 1 semesters
testX=lapply(ts1,function(x) (n-x):n)
```
If one puts this test indices in `folds`, `xgb.cv` will perform overfittinng using only test data of different tails.
One simple solution would be just changing `unlist(folds[-k])` for `setdiff(1:nrow(dall),folds[[k]])`:
```{r}
bst_folds <- lapply(seq_along(folds), function(k) {
dtest <- slice(dall, folds[[k]])
#
dtrain <- slice(dall, setdiff(1:nrow(dall),folds[[k]]))
#
handle <- xgb.Booster.handle(params, list(dtrain, dtest))
list(dtrain = dtrain, bst = handle, watchlist = list(train = dtrain,
test = dtest), index = folds[[k]])
})
```
Kindly,
Alex

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.