WikiExtractor / WikiExtractor/wikiextractor
Get all revisions content
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4k
- Forks
- 1k
- Avg merge
- 1m
- Merged PRs (30d)
- 10
Description
Hi!
As far as I understand (and tried the code), the current implementation assumes that the input dump file contains a single revision per pageID.
The historical dump files contain all revisions of a single page, and when this is given as input for the code, it generates long textual content without splitting it into revisions.
Is there a simple way to "force" the code to take into account the different revisions per pageID?
thank you!
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No file, test, or entry point is named. Start by tracing how historical dump input is parsed and how pageID content is accumulated, then compare that behavior with the dump's revision structure. Done means historical input produces separate content for each revision rather than one combined text block.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100