WikiExtractor/wikiextractor

View on GitHub

A tool for extracting plain text from Wikipedia dumps

Stars: 4k
Forks: 1k
Open beginner issues: 0
Indexed issues: 84
Avg merge: 1m
Merged PRs (30d): 10

Dominant language: Python
License: AGPL-3.0
Last GitHub push: Sep 6, 2026
Latest indexed: Sep 20, 2026
Contributing guide: No contributing guide
Code of conduct: No code of conduct
Beginner labels: No beginner labels indexed

Beginner friendly Everything

Open All

84 open issues indexed

More extensible redirect keywords Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

WikiExtractor/wikiextractor#450 · Opened Sep 6, 2026
WikiExtractor could use the language defined in the template dump, if available Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

WikiExtractor/wikiextractor#444 · Opened Sep 4, 2026
Stability - have the main process check worker and reducer health Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

WikiExtractor/wikiextractor#438 · Opened Aug 17, 2026
`#ifexist` issues - perhaps cache the existence of pages at the start? Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

WikiExtractor/wikiextractor#426 · Opened Aug 9, 2026
If a `<div>` tag has `style="display:none"` in it, perhaps that entire content should be vaporized Open

Difficulty 3/5 1-2 days Newbie friendliness 55/100

WikiExtractor/wikiextractor#421 · Opened Aug 8, 2026
`#invoke` dispatch doesn't recognize localized aliases (e.g. Urdu's `#استدعا`) Open

Difficulty 4/5 3-5 days Newbie friendliness 55/100

WikiExtractor/wikiextractor#414 · 1 comment · Opened Aug 6, 2026
Lua interpreter? Open

Difficulty 5/5 Over a week Newbie friendliness 30/100

WikiExtractor/wikiextractor#413 · Opened Aug 5, 2026
`<table>` tag removal leads to connected text? Open

Difficulty 3/5 1-2 days Newbie friendliness 45/100

WikiExtractor/wikiextractor#385 · Opened Aug 1, 2026
📋 Documentation Enhancement Suggestion Open

Difficulty 3/5 1-2 days Newbie friendliness 45/100

WikiExtractor/wikiextractor#352 · Opened Mar 26, 2026
Add man support Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

WikiExtractor/wikiextractor#351 · Opened Feb 27, 2026
3.0.6 (pypi) not fully extracting? Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

WikiExtractor/wikiextractor#346 · Opened Aug 21, 2025
Is this project abandoned? Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

WikiExtractor/wikiextractor#335 · 9 comments · 13 reactions · Opened Jun 26, 2024
OSS-Fuzz Integration Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

WikiExtractor/wikiextractor#334 · Opened May 30, 2024
Get all revisions content Open

Difficulty 5/5 Over a week Newbie friendliness 30/100

WikiExtractor/wikiextractor#332 · Opened May 20, 2024
How to store a document in a separate txt file instead of a single txt file containing multiple documents Open

Difficulty 3/5 1-2 days Newbie friendliness 30/100

WikiExtractor/wikiextractor#328 · 2 comments · Opened Mar 12, 2024
Wikidata Extraction Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

WikiExtractor/wikiextractor#325 · Opened Nov 26, 2023
Parsing seems to exclude some part of the page Open

Difficulty 3/5 1-2 days Newbie friendliness 25/100

WikiExtractor/wikiextractor#324 · Opened Nov 10, 2023
Bullet points are missing in the final extracted text Open

Difficulty 3/5 1-2 days Newbie friendliness 68/100

WikiExtractor/wikiextractor#321 · 1 comment · Opened Sep 8, 2023
Add feature to extractPage to also dump the extracted page to json/csv/txt Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

WikiExtractor/wikiextractor#317 · Opened Aug 15, 2023
Why was --keep_tables removed? Open

Difficulty 1/5 Under an hour Newbie friendliness 25/100

WikiExtractor/wikiextractor#308 · Opened Apr 8, 2023
How to extract lists pages? Open

Difficulty 3/5 1-2 days Newbie friendliness 28/100

WikiExtractor/wikiextractor#302 · Opened Dec 29, 2022
Non-textual elements score and mapframe are not filtered out Open

Difficulty 3/5 1-2 days Newbie friendliness 45/100

WikiExtractor/wikiextractor#301 · Opened Dec 29, 2022
Various tags such as q, br, ins, del are not fitered out Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

WikiExtractor/wikiextractor#300 · 1 comment · Opened Dec 29, 2022
Option to drop section titles/headers Open

Difficulty 3/5 1-2 days Newbie friendliness 42/100

WikiExtractor/wikiextractor#293 · 1 comment · 1 reaction · Opened Sep 19, 2022
ModuleNotFoundError: No module named '__main__.extract'; '__main__' is not a package Open

Difficulty 4/5 3-5 days Newbie friendliness 20/100

WikiExtractor/wikiextractor#291 · 1 reaction · Opened Aug 4, 2022
how to get mention/anchor by wikiextractor. Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

WikiExtractor/wikiextractor#286 · Opened Apr 3, 2022
Question: Cirrus Extractor vs. "normal" Extractor - who creates cleaner texts? Open

Difficulty 1/5 Under an hour Newbie friendliness 15/100

WikiExtractor/wikiextractor#282 · 1 comment · Opened Feb 21, 2022
KeyError for producing HTML output with `--html` Open

Difficulty 3/5 1-2 days Newbie friendliness 30/100

WikiExtractor/wikiextractor#280 · 1 comment · 9 reactions · Opened Dec 18, 2021
Extraction of french wikipedia has lacks and syntaxic faults Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

WikiExtractor/wikiextractor#279 · 1 comment · 2 reactions · Opened Dec 8, 2021
文章摘要抽取不到数据 Open

Difficulty 3/5 1-2 days Newbie friendliness 25/100

WikiExtractor/wikiextractor#277 · 4 comments · Opened Nov 8, 2021
templates are not extracted correctly Open

Difficulty 3/5 1-2 days Newbie friendliness 64/100

WikiExtractor/wikiextractor#275 · 1 comment · 1 reaction · Opened Oct 15, 2021
Would be nice to continue after interrupt Open

Difficulty 3/5 1-2 days Newbie friendliness 42/100

WikiExtractor/wikiextractor#270 · Opened Aug 16, 2021
[Feature Request]: Capture Paragaph Heading information Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

WikiExtractor/wikiextractor#268 · 1 comment · Opened Aug 4, 2021
"revid" is incorrectly the page.revision.contributor.id, when it should be the page.revision.id Open

Difficulty 3/5 1-2 days Newbie friendliness 45/100

WikiExtractor/wikiextractor#267 · 2 comments · Opened Aug 1, 2021
How to preserve sections? Open

Difficulty 3/5 1-2 days Newbie friendliness 30/100

WikiExtractor/wikiextractor#264 · 1 comment · 5 reactions · Opened Jun 29, 2021
section arg not regonized Open

Difficulty 3/5 1-2 days Newbie friendliness 25/100

WikiExtractor/wikiextractor#255 · 1 comment · Opened May 3, 2021
Problems with the import extract Open

Difficulty 2/5 1-3 hours Newbie friendliness 35/100

WikiExtractor/wikiextractor#252 · 2 comments · Opened Apr 2, 2021
the function dropNested Open

Difficulty 2/5 1-3 hours Newbie friendliness 35/100

WikiExtractor/wikiextractor#248 · Opened Mar 16, 2021
Help! DO dump files contain the wikitable in the wikipedia? Open

Difficulty 3/5 1-2 days Newbie friendliness 25/100

WikiExtractor/wikiextractor#247 · 1 comment · Opened Feb 27, 2021
Missing bullets content Open

Difficulty 3/5 1-2 days Newbie friendliness 42/100

WikiExtractor/wikiextractor#245 · 1 comment · 6 reactions · Opened Feb 9, 2021
TypeError: __init__() missing 2 required positional arguments: 'title' and 'page' Open

Difficulty 4/5 3-5 days Newbie friendliness 20/100

WikiExtractor/wikiextractor#240 · Opened Dec 22, 2020
bugs, syntax errors, documentation errors and wild popularity Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

WikiExtractor/wikiextractor#231 · 2 comments · Opened Dec 1, 2020
for people with all kinds of weird bugs Open

Difficulty 1/5 Under an hour Newbie friendliness 35/100

WikiExtractor/wikiextractor#222 · 7 comments · 5 reactions · Opened Oct 13, 2020
[For reference] Another Wiki extractor tool Open

Difficulty 5/5 Over a week Newbie friendliness 10/100

WikiExtractor/wikiextractor#220 · Opened Sep 17, 2020
How to extract articles from main namespace only Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

WikiExtractor/wikiextractor#213 · 1 comment · Opened Jul 22, 2020
Wrong guide for the usage for the argument “--no-templates” Open

Difficulty 1/5 Under an hour Newbie friendliness 45/100

WikiExtractor/wikiextractor#209 · 1 comment · 2 reactions · Opened Jun 3, 2020
Can i have 1 output per file Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

WikiExtractor/wikiextractor#206 · 2 comments · Opened Apr 14, 2020
UnicodeDecodeError when extracting from an (decompressed) .xml dump Open

Difficulty 2/5 1-3 hours Newbie friendliness 52/100

WikiExtractor/wikiextractor#200 · 1 comment · 3 reactions · Opened Jan 8, 2020
template expansion works well in an old version Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

WikiExtractor/wikiextractor#197 · 2 comments · 1 reaction · Opened Nov 18, 2019
Last category in article not checked correctly Open

Difficulty 2/5 1-3 hours Newbie friendliness 42/100

WikiExtractor/wikiextractor#193 · 1 comment · 1 reaction · Opened Oct 26, 2019

WikiExtractor/wikiextractor

Get new issues in your inbox