scrapinghub / scrapinghub/dateparser

Dataset of article publication dates as they appear on the web

Open
#928 1 comment 8 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
2.9k
Forks
520
Avg merge
22h 56m
Merged PRs (30d)
6

Description

Hi, on behalf of Automatic Extraction team from Zyte, I'd like to thank dateparser developers for a great library, and share a dataset of article publication dates as they appear on the web - I think it can be useful for the library development. It was collected on Oct 2020 - Mar 2021, has 300k rows, and contains the following columns:

Here are the first few rows:

suffix articleLanguage webPageLanguages datePublishedRaw
eu en en lg October 22, 2020
es es es ca 2020-10-09T17:53:37+02:00
fr fr fr en 02/10/2020 14:53:07
ie en en 2020-10-19T12:54:52Z
com en en th 28 Oct 2020 at 13:50

And here it dataset in full: article_date_sample_Oct_2020_Mar_2021_public.csv.zip

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the attached article_date_sample_Oct_2020_Mar_2021_public.csv.zip and the listed dataset columns. The issue does not name a target file, test, or concrete code change; completion would require deciding how, if at all, this 300k-row dataset should support dateparser development.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.