ArchiveBox / ArchiveBox/abx-plugins
Feature Request: add BDfR as a new extractor for archiving Reddit content
- Dominant language
- Python
- Stars
- 14
- Forks
- 3
- Avg merge
- 3h 12m
- Merged PRs (30d)
- 15
Description
### Discussed in https://github.com/ArchiveBox/ArchiveBox/discussions/754
Originally posted by **BlipRanger** May 24, 2021
Just wanted to make a quick mention of [BDfR](https://github.com/aliparlakci/bulk-downloader-for-reddit) as a cool project that might make for a good starting point for the unrolling of reddit comments/posts as mentioned in the roadmap. They currently support grabbing a variety of media types from the post as well as the comments/text in a separate (json) file. I've been working on an [addon](https://github.com/BlipRanger/bdfr-html) for it lately and I think it's a pretty great project with well-maintained code. If nothing else, they have really good examples of working with reddit data which could be useful! Just wanted to bring that to your attention!
I'd love to add [BDfR](https://github.com/aliparlakci/bulk-downloader-for-reddit) as an extractor for Reddit content (and something similar for Twitter too https://github.com/ArchiveBox/ArchiveBox/issues/345) but am somewhat swamped with work and travel for the near future.
If you @BlipRanger or anyone else wants to add it as an extractor (matching the style of our other extractors, e.g. [`archivebox/extractors/media.py`](https://github.com/ArchiveBox/ArchiveBox/blob/dev/archivebox/extractors/media.py) is a great example to copy), I'd be happy to review PRs!
We have some good instructions for contributing a new extractor and getting started with ArchiveBox development in general:
- https://github.com/ArchiveBox/ArchiveBox/blob/dev/README.md#contributing-a-new-extractor
- https://github.com/ArchiveBox/ArchiveBox/blob/dev/README.md#archivebox-development
- https://github.com/ArchiveBox/ArchiveBox/blob/dev/.github/CONTRIBUTING.md
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the new-extractor instructions in README.md and review archivebox/extractors/media.py as the stated implementation example. Check the ArchiveBox development and CONTRIBUTING guidance before examining how BDfR handles Reddit posts, comments, and media. Done means a reviewed extractor that archives Reddit content in the project’s existing extractor style.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100