Buffer dtype mismatch, expected 'int' but got 'long'
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 5.1k
- Forks
- 724
- PR merge metrics
- No merged PRs in 30d
Description
`
File "/opt/ml/processing/input/code/train.py", line 934, in output_dir=args.output_dir File "/opt/ml/processing/input/code/train.py", line 903, in run_model output_dir File "/opt/ml/processing/input/code/train.py", line 867, in run_lightfm_model num_threads=num_threads, File "/opt/ml/processing/input/code/train.py", line 832, in fit_model num_threads=num_threads, File "/usr/local/lib/python3.7/site-packages/lightfm/lightfm.py", line 540, in fit verbose=verbose, File "/opt/ml/processing/input/code/train.py", line 755, in fit_partial verbose, File "/usr/local/lib/python3.7/site-packages/lightfm/lightfm.py", line 644, in fit_partial self.loss, File "/usr/local/lib/python3.7/site-packages/lightfm/lightfm.py", line 668, in _run_epoch self._get_positives_lookup_matrix(interactions) File "lightfm/_lightfm_fast_openmp.pyx", line 167, in lightfm._lightfm_fast_openmp.CSRMatrix.__init__ | Traceback (most recent call last): File "/opt/ml/processing/input/code/train.py", line 934, in output_dir=args.output_dir File "/opt/ml/processing/input/code/train.py", line 903, in run_model output_dir File "/opt/ml/processing/input/code/train.py", line 867, in run_lightfm_model num_threads=num_threads, File "/opt/ml/processing/input/code/train.py", line 832, in fit_model num_threads=num_threads, File "/usr/local/lib/python3.7/site-packages/lightfm/lightfm.py", line 540, in fit verbose=verbose, File "/opt/ml/processing/input/code/train.py", line 755, in fit_partial verbose, File "/usr/local/lib/python3.7/site-packages/lightfm/lightfm.py", line 644, in fit_partial self.loss, File "/usr/local/lib/python3.7/site-packages/lightfm/lightfm.py", line 668, in _run_epoch self._get_positives_lookup_matrix(interactions) File "lightfm/_lightfm_fast_openmp.pyx", line 167, in lightfm._lightfm_fast_openmp.CSRMatrix.__init__
`
After looking, I guess large sparse matrices are not supported and this is basically happening due to the handling of large_indices directly in Cython.
The interactions matrix shape is (10.000.000, 190.000)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The report points to lightfm/_lightfm_fast_openmp.pyx:167 and an interactions matrix shaped (10,000,000, 190,000); start by reproducing the dtype mismatch with that scale. Inspect CSRMatrix.__init__ and the surrounding fit_partial path, then verify that training handles the large-index case without raising the reported error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100