Error in cox-regression while evaluating
- Dominant language
- C++
- Stars
- 28.8k
- Forks
- 8.9k
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 54
Description
I am using XGBoost version 1.3.3 on Windows with Python 3.6.8. On attempting to do training with objective set to `survival:cox`, I repeatedly get this error.
Traceback (most recent call last):
File "xgboost_survival_cv.py", line 94, in
evals=[(data_m, 'train'), (data_v, 'eval')])
File "C:\Program Files\Python36\lib\site-packages\xgboost\training.py", line 235, in train
early_stopping_rounds=early_stopping_rounds)
File "C:\Program Files\Python36\lib\site-packages\xgboost\training.py", line 110, in _train_internal
if callbacks.after_iteration(bst, i, dtrain, evals):
File "C:\Program Files\Python36\lib\site-packages\xgboost\callback.py", line 427, in after_iteration
self._update_history(score, epoch)
File "C:\Program Files\Python36\lib\site-packages\xgboost\callback.py", line 393, in _update_history
name, s = d[0], float(d[1])
ValueError: could not convert string to float: '-nan(ind)'
**There are no `nan` in my data**
I initially assumed this might be due to some overflow since this error often appeared when the test loss (`cox-nloglik`) exceeded 15 or so (in the last successful boosting iteration), as confirmed by the disappearance of the error on using lesser boosting rounds, smaller learn rate, smaller trees (no overfitting and hence no blowup of the test loss?) or switching off evaluation (empty evals list). But, later, I got the same error when the test loss was 6 (in the last successful boosting iteration). Further, on removing the evaluation (I need to implement `early_stopping_rounds` so this is not a long-term option), I still get `nan` (or `inf`) in the prediction output, though no error. The data is highly censored (90% right censored), in case that matters.
The run parameters were thus:
```
{'colsample_bytree': 0.8, 'eta': 0.1, 'max_delta_step': 0, 'max_depth': 3, 'min_child_weight': 100,
'num_parallel_tree': 20, 'sampling_method': 'uniform', 'subsample': 0.8, 'tree_method':'gpu_hist',
'verbosity':1, 'seed':0, 'objective':'survival:cox', 'eval_metric':'cox-nloglik'}
```
The same error for many different parameter sets. Another example:
` {'colsample_bytree': 0.8, 'eta': 0.3, 'max_delta_step': 0, 'max_depth': 3, 'min_child_weight': 100, 'num_parallel_tree': 1, 'sampling_method': 'gradient_based', 'subsample': 0.2}`
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.