dmlc / dmlc/xgboost

reading some 'nan' from libsvm file leads to label leakage

Open
#4,356 8 comments 1 reaction 1 assignee Claimed by @trivialfis View on GitHub
type: bug
Dominant language
C++
Stars
28.8k
Forks
8.9k
Avg merge
1d 12h
Merged PRs (30d)
54

Description

Reading a libsvm file where some points have 'nan' values leads to surprising results.

1. The DMatrix read by xgboost is different than one read by `sklearn.datasets.load_svmlight_file` when the file contains 'nan'.
Version from `sklearn.datasets.load_svmlight_file` yields the same result for 'nan' as for missing values.
2. When training using such dataset we surprisingly yield better results in terms of ndcg.

I prepared a snippet that reproduces this issue which is runnable under google collab:

https://gist.github.com/kretes/ae8a9948af7d68cb7f8c0757a44ddfd9

You can see there that results obtained by reading 'nan' file directly by xgb are different that the ones obtained from 'empty' file, while reading 'nan' file with `sklearn.datasets.load_svmlight_file` treats nan the same as missing

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.