microsoft / microsoft/qlib

numpy.datetime64 precision cause dict indexing failure in index_data.py

Open
#1,806 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
48.7k
Forks
7.7k
PR merge metrics
No merged PRs in 30d

Description

🐛 Bug Description

ns precision numpy.datetime64 is equal to second precision
however when doing indexing of a dict keys, it will cause failure

numpy.datetime64('2017-01-04T00:00:00.000000000')==numpy.datetime64('2017-01-04T00:00:00')
Out[24]: True
self.index_map[numpy.datetime64('2017-01-04T00:00:00.000000000')]
Out[25]: 1
self.index_map[numpy.datetime64('2017-01-04T00:00:00')]
Traceback (most recent call last):
File "F:\work\env\py311\Lib\site-packages\IPython\core\interactiveshell.py", line 3508, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "", line 1, in
self.index_map[numpy.datetime64('2017-01-04T00:00:00')]
~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: numpy.datetime64('2017-01-04T00:00:00')

it occurs at qlib/qlib/utils/index_data.py, line 157:
try:
return self.index_map[self._convert_type(item)]
except IndexError as index_e:
raise KeyError(f"{item} can't be found in {self}") from index_e

maybe I did something wrong, hoping for help

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in qlib/qlib/utils/index_data.py around line 157 and reproduce the reported dictionary lookups with the two numpy.datetime64 precisions shown in the issue. Determine the intended behavior for equivalent timestamps and verify that the resulting lookup behavior is consistent, or that any intentional limitation is documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.