scrapinghub / scrapinghub/dateparser

Parsing relative dates error with Taiwanese Mandarin

Open
#913 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Type: Bug - Language
Dominant language
Python
Stars
2.9k
Forks
520
Avg merge
22h 56m
Merged PRs (30d)
6

Description

I have trouble parsing the following string "2分鐘前". It should translate to "2 minutes ago" in Mandarin (checked with DeepL, GoogleTranslate, etc), but parsing returns None.

>>> dateparser.parse("2分鐘前")
>>> dateparser.parse("2 分鐘前")
datetime.datetime(2021, 4, 28, 20, 45, 19, 266428)

I added a space in between the number and the symbols and it works, but ideally I would like it to work without adding it.
I think it may be because I got it from a Taiwanese version of a website, but I am no expert.

I translated back to Mandarin "2 minutes ago" and got "2分钟前" (notice the symbol in the middle is different), which parses fine with and without a space, but it'd be great if the ?Taiwanese? version will work as well.

>>> dateparser.parse("2分钟前")
datetime.datetime(2021, 4, 28, 20, 45, 32, 129967)
>>> dateparser.parse("2 分钟前")
datetime.datetime(2021, 4, 28, 20, 45, 35, 659334)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the reported dateparser.parse examples for “2分鐘前”, “2 分鐘前”, and “2分钟前”. Trace the Taiwanese Mandarin locale handling from that entry point and add or update the relevant test; done means the no-space Taiwanese form parses successfully without regressing the existing Mandarin forms.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
localization
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.