mozilla / mozilla/readability

[bug] Single word Wikipedia headers are removed while multiple words are parsed

Open
#998 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
11.5k
Forks
732
PR merge metrics
No merged PRs in 30d

Description

Hi,

For context, this is an issue I noticed on my karakeep instance, a FOSS bookmarking solution. I opened an issue there and was told the bug might be on readability's side. So I opened firefox, opened the webpage and indeed confirmed the bug is caused by readability.


Describe the Bug

I noticed that saving wikipedia pages to karakeep would sometimes miss the section headers.

I finally figured out how to reproduce this: headers that are only a single world are always removed whereas those that span over multiple words are kept.

This might happen outside of wikipedia too but I can't know what I'm missing.

Steps to Reproduce
  1. On firefox, go to German tank problem.
  2. Notice its table of content and how some sections are single words like Suppositions while others are over multiple words like Historical example of the problem.
Image
  1. Open readability on that page
  2. Search each of the table of content section headers. Presents are Historical example of the problem, Frequentist analysis and Bayesian Analysis. Also See also, Further reading and External links. Missing are: Suppositions, Example, Countermeasures, Notes and References.
  3. Notice that the common feature of the missing title is the fact that they are single words.
Expected Behaviour

All headers should be kept.

Screenshots or Additional Context

Here is the table of content:

Image

And here notice the missing Suppositions section header:

Image Image

But other section headers do exist:

Image
Device Details

No response

Exact Firefox Version

144.0

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the issue with the German tank problem Wikipedia page in Firefox and compare the retained and missing section headers. Trace the header parsing and filtering behavior responsible for single-word titles; done means all listed headers, including Suppositions, Example, and References, are retained.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
web-dev
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.