numpy / numpy/numpy

“TypeError: data type not understood” comparing dtype np.datetime64 with issubdtype

Open
#15,770 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

33 - Question component: numpy.datetime64
Dominant language
Python
Stars
32.8k
Forks
12.8k
Avg merge
1d 7h
Merged PRs (30d)
197

Description

Reproducing code example:
import pandas as pd
import numpy as np
df_log = pd.DataFrame(columns=['ts', 'event'])
ts_start = 1582582300
ts_end = ts_start + 30
df_log['ts']=pd.Series(np.arange(ts_start, ts_end))
df_log.loc[df_log['ts']==1582582300 + 3, 'event']="Entered room"
df_log.loc[df_log['ts']==1582582300 + 12, 'event']="Door opened"
df_log.loc[df_log['ts']==1582582300 + 18, 'event']="Door closed"
df_log.loc[df_log['ts']==1582582300 + 25, 'event']="Left room"
df_log['ts']=pd.to_datetime(df_log["ts"], unit='s', utc=True).dt.tz_convert('US/Central')
for i in range(len(df_log.dtypes)):
    dtype = df_log.dtypes[i]
    print("testing:", dtype)
    if np.issubdtype(dtype, np.datetime64):
         print("is subtype")
Error message:
testing: datetime64[ns, US/Central]
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-2-225a4521e525> in <module>
     13     dtype = df_log.dtypes[i]
     14     print("testing:", dtype)
---> 15     if np.issubdtype(dtype, np.datetime64):
     16          print("is subtype")

/usr/local/lib/python3.7/site-packages/numpy/core/numerictypes.py in issubdtype(arg1, arg2)
    391     """
    392     if not issubclass_(arg1, generic):
--> 393         arg1 = dtype(arg1).type
    394     if not issubclass_(arg2, generic):
    395         arg2_orig = arg2

TypeError: data type not understood
Numpy/Python version information:
print(numpy.__version__, sys.version)
1.18.1 3.7.6 (default, Dec 30 2019, 19:38:26) 
[Clang 11.0.0 (clang-1100.0.33.16)]

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the reproducer with NumPy 1.18.1 and Python 3.7.6, focusing on the np.issubdtype call with pandas' timezone-aware datetime64 dtype. Inspect NumPy's dtype handling around the numerictypes.py path shown in the traceback and compare behavior with timezone-naive dtypes. Done means the reported comparison has defined, tested behavior instead of raising the shown TypeError.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, pandas, python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.