Data loss converting from scipy.sparse.csr_matrix to xgboost.DMatrix
- Dominant language
- C++
- Stars
- 28.8k
- Forks
- 8.9k
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 54
Description
scipy csr matrices have an implied value of zero for all non specified values. However this isn't being respected when converting between the two i.e:
```
from scipy.sparse import csr_matrix
from xgboost import DMatrix
raw_data = [
[1, 0, 2],
[0, 3, 0],
[4, 0, 0]
]
sp_csr = csr_matrix(raw_data)
dmatrix = DMatrix(raw_data)
print(dmatrix.get_data())
# Coords Values
# (0, 0) 1.0
# (0, 1) 0.0
# (0, 2) 2.0
# (1, 0) 0.0
# (1, 1) 3.0
# (1, 2) 0.0
# (2, 0) 4.0
# (2, 1) 0.0
# (2, 2) 0.0
print(DMatrix(sp_csr).get_data())
# Coords Values
# (0, 0) 1.0
# (0, 2) 2.0
# (1, 1) 3.0
# (2, 0) 4.0
```
I would expect the two 'print' statements to return equivalent results, but the zeroes are instead being treated as missing values.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the conversion with scipy.sparse.csr_matrix and DMatrix, then compare get_data() for dense and CSR inputs. Done means explicitly stored zero values are preserved consistently instead of being treated as missing values.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100