Missing first article's section for decolonizepalestine.com/myth/a-land-...

Open
#948 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
38/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
javascript
Domain
content

Research direction

Reproduce the extraction with the provided URL, axios, JSDOM, and Readability.parse() entry point, then inspect why the first article section is omitted. Add a regression case for this page and verify that the parsed text includes the opening section as well as the later content.

Written by the indexing model from the issue text.

Description

Hello.

For some reason, readablity is missing the content of the first section of the page: decolonizepalestine/myth/a... :

“A land without a people for a people without a land”

It is almost impossible to acquaint yourself with scholarship on Palestine, or be involved in Palestinian activism to any extent without coming across a variation of the above. Sadly, despite all the meticulous scholarship of Palestinians and others on the topic, variations of this talking point remain a prominent feature of Israeli propaganda. But how cou
...
..
..

Here is the code. You see that already tried charThreshold and nbTopCandidates:

const { data } = await axios.get( url ); // https://decolonizepalestine.com/myth/a-land-without-a-people-for-a-people-without-a-land/
const dom = new JSDOM( data, { url });

const reader = new Readability( dom.window.document, { charThreshold: 40, nbTopCandidates: 20 });
const article = reader.parse();

the textContent start by : This slogan persists to ( the second section )

versions:

  "dependencies": {
    "@mozilla/readability": "^0.5.0",
    "axios": "^1.7.9",
    "eslint": "^9.17.0",
    "jsdom": "^26.0.0"
  }

It happens in the firefox itself as well:

image

image

Dominant language
JavaScript
Stars
11.5k
Forks
732
PR merge metrics
No merged PRs in 30d

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from mozilla/readability

All issues in mozilla/readability

Similar issues

More JavaScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.