pydata / pydata/sparse

dtype: NotImplementedError: float16

Open
#263 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

upstream
Dominant language
Python
Stars
668
Forks
141
Avg merge
2d 8h
Merged PRs (30d)
4

Description

Hello, I am trying to reduce the memory footprint of the matrices containing small integers, and want to set the dtype to np.int8 or np.float16.

import sparse
import numpy as np
x=sparse.COO(np.random.rand(5*10**2, 5*10**2)).astype(np.float16)
y = sparse.dot(x,x)
z = sparse.dot(y,y)

This is the error I get:

myenvpath/lib/python3.6/site-packages/sparse/coo/common.py in _dot_coo_coo_type(dt1, dt2)
   1208 
   1209     @numba.jit(nopython=True, nogil=True,
-> 1210                locals={'data_curr': numba.numpy_support.from_dtype(dtr)})
   1211     def _dot_coo_coo(coords1, data1, coords2, data2):  # pragma: no cover
   1212         """

myenvpath/lib/python3.6/site-packages/numba/numpy_support.py in from_dtype(dtype)
    106             return types.NestedArray(subtype, dtype.shape)
    107 
--> 108     raise NotImplementedError(dtype)
    109 
    110 

NotImplementedError: float16

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with sparse/coo/common.py at _dot_coo_coo_type and reproduce the reported sparse.COO and sparse.dot sequence using np.float16 or np.int8. Check how the dtype reaches the Numba helper; done means the reported dot operations no longer raise NotImplementedError for the requested dtype.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.