scrapinghub / scrapinghub/dateparser

Handling whitespace when matching relative dates

Open
#916 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Type: Bug - Language
Dominant language
Python
Stars
2.9k
Forks
520
Avg merge
22h 56m
Merged PRs (30d)
6

Description

This is for version dateparser==1.0.0.
I've noticed that for some of the dates I tried to parse I need to add spaces to make parsing work and for some I need to remove them.
It seems a bit random and I am guessing it depends on how the string was added to date_translation_data.
That doesn't seem very flexible or reliable, and I wonder if it would make sense to either remove all whitespace from both input data and date_translation_data before matching, or make them optional.

Examples

  • 2 minutes ago in Mandarin
>>> dateparser.parse("2分鐘前")
>>> dateparser.parse("2 分鐘前")                  -> works only if you add space between the number and symbol
datetime.datetime(2021, 4, 28, 20, 45, 19, 266428)
  • 2 months ago in Thai
>>> dateparser.parse("2 เดือน ที่ผ่านมา")            -> doesn't work with space between months and ago
>>> dateparser.parse("2 เดือนที่ผ่านมา")             -> works when you remove space between months and ago
datetime.datetime(2021, 2, 28, 22, 3, 40, 264496)
>>> dateparser.parse("2เดือนที่ผ่านมา")              -> doesn't work when you remove all spaces

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the Mandarin and Thai examples with dateparser.parse, then inspect how whitespace in date_translation_data is matched. Done means the shown variants parse consistently without manually adding or removing spaces, with coverage for the reported examples.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.