numpy / numpy/numpy

np.issubdtype does not work well with pandas

Open
#9,474 15 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

15 - Discussion component: numpy.dtype
Dominant language
Python
Stars
32.8k
Forks
12.8k
Avg merge
1d 7h
Merged PRs (30d)
197

Description

For example, below I am trying to use np.issubdtype to check if a pandas categorical is a subtype of a np.number. IMO this should definitely be false rather than throwing an error.

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-99-4ba76ca46d4a> in <module>()
     32 
---> 33 np.issubdtype(df.dtypes[1], np.number)

/Users/jhoward/homebrew/lib/python2.7/site-packages/numpy/core/numerictypes.pyc in issubdtype(arg1, arg2)
    753     """
    754     if issubclass_(arg2, generic):
--> 755         return issubclass(dtype(arg1).type, arg2)
    756     mro = dtype(arg2).type.mro()
    757     if len(mro) > 1:

TypeError: data type not understood

This issue should be fixed on numpy's side because external types should by default compare as false unless the external package developers have explicitly used the numpy type system.

If this is a reasonable update, I would be glad to fix this myself.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with np.issubdtype and the dtype handling shown in numpy/core/numerictypes.pyc, then inspect the current implementation and related tests. Reproduce the pandas categorical example and verify that comparing an unsupported external dtype returns false rather than raising TypeError.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, pandas, python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.