scrapinghub / scrapinghub/dateparser
Dateparsed hangs minutes on certain dates
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.9k
- Forks
- 520
- Avg merge
- 22h 56m
- Merged PRs (30d)
- 6
Description
I don't have much details atm but I noticed that in certain cases the performance is awful. I wouldn't say it's a specific version, it has been always like that though rare. I took an eye on it.
E.g.
188.09s call manhattan_hub/helpers/tests/test_time_helper.py::test_extract_datetime_from[: 20/05/20-2020-05-20-mmdd]
@pytest.mark.parametrize("ocr_date, parsed, date_format", ocr_to_iso)
def test_extract_datetime_from(ocr_date, parsed, date_format):
with freeze_time("2021-12-02"):
assert (
str(
time_helper.extract_datetime_from(
ocr_date,
date_format=date_format,
ignore_dates_older_than_years=10,
max_future_date=time_helper.x_days_in_future(12 * 31),
)
).split(" ")[0]
== parsed
)
The tests are launched in parallel, the code deployed in Django environment with multiple wsgi workers.
Python 3.7.13
dateparser 1.1.8
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the slow case in manhattan_hub/helpers/tests/test_time_helper.py::test_extract_datetime_from, using the 20/05/20 input and the shown date format and date limits. Trace time_helper.extract_datetime_from and dateparser behavior to identify why this case takes minutes, then add regression coverage showing the case completes promptly without changing the expected parsed date.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100