scrapinghub / scrapinghub/dateparser

Compreshansive keywords processing for relative descriptions

Open
#344 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Type: Enhancement
Dominant language
Python
Stars
2.9k
Forks
520
Avg merge
22h 56m
Merged PRs (30d)
6

Description

Relative time definitions are supported to some extent however, based on what I could see inside the code, unless there is a major overhaul it will not variety of relative descriptions:

Here I am putting a list for exhaustive definitions that are relatively common yet not supported:

1. weekday/weekend

  • While we can say 'Saturday' pointing to upcoming Saturday - but "weekend"; isn't support. Also, term "in 3 week" shows period of 1 week - but weekend should show strictly "2 days" as period.
  • Further, Next weekday (currently not supported) refers to next day for Monday to Thursday, but it will refer to Monday if for saturday and sunday - hence this can't be done simply by adding more definitions in language simplification section. these keywords should be processed in the core.

2. Next/ Last

  • next sunday, next monday. etc. not yet supported. same is true for next january, next april etc.
  • Though "next week" and "next year" is supported - the interpretation is actually wrong. "Next week" should point to start of the next week with period of 1 week where as currently it just adds 7 days from today. [ e.g. if I say (on Wednesday) let's do it next week - the correct period from coming Monday to Sunday and not from Wednesday to next tuesday. Same is true for "next month" and "next year".
  • other element that should be supported is "next to next" and "last to last"

3. before/after/from/prior to/post

  • date can be expressed as "X beforeY" or "X afterY" - where X can be any time period (e.g. x weeks, x days, weekend i.e. weekend before "") and Y can be any other date that can be parsed on its own. If we just specify before Y than the output should be Y only but output can somehow show direction indicating or .
  • key word "from" can be equivalent to "after" for future dates and "before" for past dates. 2 days from yesterday is -3 days. 2 days from tomorrow is +3 days.
  • after and before also imply the start/end of the period. For example: "after next week" is a date after end of next of week, where as "before last week" is a date prior to start of the last week.
  • Terms "prior to" and "post" will be direct synonym of "before" and "after"

4. this/coming/current

  • the term <this/coming> can be skipped in many scenarios - such as "this sunday" or "coming sunday" is same as "sunday". However, "this week" should point start of the week - with period of 7 days. same way "this year" should point to start of the year with period =1 year.
  • Interestingly there are other important relative meaning that needs to be understood.
    "coming sunday" and "this sunday" is same as "sunday" and "next sunday" is a sunday after that. However, "coming year" is typically same as "next year" not "current/this year".

5. week of month/day of week/day of year/week of year

  • we can specify things like 2nd week of August. It may be invalid to say 2nd week of Sunday or 1st week of 31st.
  • similarly we can specify 3rd day of next week or 2nd day of last week. similarly 4th day of 2017 is valid and so is 21st weekend of 2016.

6. start of/end of

  • Term "start of" refers to the first date of any period and "end of" refers to last one. start of the year is 1st Jan of that year, and end of the year is 31st December. Same applies to start of the next week, or next moth, end of month etc.

7 Periods

  • string such as "1 month", "three weeks", "4 days", "6 weeks", "3 years" etc. doesn't return absolute time but should return a period such as relativedelta. If needed it can be added to expected reference to make it absolute time by the application.
  • it can be composite as well such as "1 year 3 months", "3 month 2 weeks" etc.
  • periods also applies to time i.e. "3 hours", "5 minutes", "19 hours 32 mins" etc.

8 named timebands

  • define terms like morning, evening, night, late evening, early morning. typically I could define morning as either 7:00 am or 9:00am in simplifcations sections. However that is not appropriate definition. I would like to define morning as a time period 7:00 am to 12:00 am (or to be precise 7:00 am with 5 hours as period). Similar definitions can be applied for other day parts. This will allow terminology like "tomorrow morning", 31st night, Saturday evening etc.

Summary: Some examples that works by using compounds of this
next week, next weekened, next sunday, "coming weekend", next January, last weekend, 3 days before next weekend, 2 days before last Thursday, coming weekend, current month, this Saturday, 5 days from tomorrow, week before last month, week after next january, 2nd week of 2017. end of next month. start of 4th week, yesterday afternoon, Sunday evening. next weekend early morning.

I am sure there would be many other variations might exist. Hence, it is important that these keywords should be parsed, processed and their meaning should be applied based on context rather than being
used under simplification framework.

I know this is really a lot of asking - but I am planning to work on some of this. So I would like all of you to pour in right design approaches. Don't want to create a patchwork that falls short of many use cases.

Can someone guide on what would be good approach?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the existing relative-description handling and the language simplifications referenced in the issue. Define the supported grammar and semantics for relative dates, periods, and named timebands before changing code; done requires an agreed design covering the listed examples and edge cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.