scrapinghub / scrapinghub/dateparser
Error handling old dates in version 0.6.0
Open
Nobody has claimed this yet.
Type: Bug
- Dominant language
- Python
- Stars
- 2.9k
- Forks
- 520
- Avg merge
- 22h 56m
- Merged PRs (30d)
- 6
Description
Sometimes a user may need to deal with old dates, e.g.
>>> datetime.datetime.strptime('1 January 0001', '%d %B %Y')
datetime.datetime(1, 1, 1, 0, 0)
In version 0.5.0 it used to be okay:
>>> dateparser.__version__
'0.5.0'
>>> dateparser.parse('1 January 0001', date_formats=['%d %B %Y'])
datetime.datetime(1, 1, 1, 0, 0)
But in version 0.6.0 it fails:
>>> dateparser.__version__
'0.6.0'
>>> dateparser.parse('1 January 0001', date_formats=['%d %B %Y'])
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/pengyu/temp/landtech/virtualenv/lib/python2.7/site-packages/dateparser/conf.py", line 84, in wrapper
File "/home/pengyu/temp/landtech/virtualenv/lib/python2.7/site-packages/dateparser/__init__.py", line 40, in parse
data = parser.get_date_data(date_string, date_formats)
File "/home/pengyu/temp/landtech/virtualenv/lib/python2.7/site-packages/dateparser/date.py", line 359, in get_date_data
parsed_date = _DateLanguageParser.parse(
File "/home/pengyu/temp/landtech/virtualenv/lib/python2.7/site-packages/dateparser/date.py", line 141, in parse_with_formats
return {'date_obj': None, 'period': period}
File "/home/pengyu/temp/landtech/virtualenv/lib/python2.7/site-packages/dateparser/utils/__init__.py", line 147, in apply_timezone_from_settings
if key not in registry_dict:
File "/home/pengyu/temp/landtech/virtualenv/lib/python2.7/site-packages/pytz/tzinfo.py", line 309, in localize
loc_dt = dt + delta
OverflowError: date value out of range
(Tested with Python 2.7.13)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the failure with Python 2.7.13 using dateparser.parse('1 January 0001', date_formats=['%d %B %Y']). Start at date.py:141 and date.py:359, then inspect utils/init.py:147 and pytz.tzinfo.py:309 in the traceback. Done means the old-date input no longer raises OverflowError and returns the expected datetime.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100