dmlc / dmlc/xgboost

Data loss converting from scipy.sparse.csr_matrix to xgboost.DMatrix

Open
#10,532 4 comments 0 reactions 0 assignees View on GitHub
feature-request type: question
Dominant language
C++
Stars
28.8k
Forks
8.9k
Avg merge
1d 12h
Merged PRs (30d)
54

Description

scipy csr matrices have an implied value of zero for all non specified values. However this isn't being respected when converting between the two i.e:
```
from scipy.sparse import csr_matrix
from xgboost import DMatrix

raw_data = [
[1, 0, 2],
[0, 3, 0],
[4, 0, 0]
]
sp_csr = csr_matrix(raw_data)
dmatrix = DMatrix(raw_data)

print(dmatrix.get_data())
# Coords Values
# (0, 0) 1.0
# (0, 1) 0.0
# (0, 2) 2.0
# (1, 0) 0.0
# (1, 1) 3.0
# (1, 2) 0.0
# (2, 0) 4.0
# (2, 1) 0.0
# (2, 2) 0.0

print(DMatrix(sp_csr).get_data())
# Coords Values
# (0, 0) 1.0
# (0, 2) 2.0
# (1, 1) 3.0
# (2, 0) 4.0
```
I would expect the two 'print' statements to return equivalent results, but the zeroes are instead being treated as missing values.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the conversion with scipy.sparse.csr_matrix and DMatrix, then compare get_data() for dense and CSR inputs. Done means explicitly stored zero values are preserved consistently instead of being treated as missing values.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.