NanmiCoder / NanmiCoder/MediaCrawler
[功能建议] 新增微信公众号文章批量采集 与 通用图文(图片+正文)采集能力
Open
Nobody has claimed this yet.
enhancement
question
- Dominant language
- Python
- Stars
- 65.3k
- Forks
- 12.6k
- PR merge metrics
- No merged PRs in 30d
Description
背景 / 动机
目前 MediaCrawler 支持 xhs/dy/ks/bili/wb/tieba/zhihu，
但缺少对「微信公众号文章」的采集，也缺少一个「通用的图文
（封面 + 多张配图 + 正文）落库」能力（当前图片仅在
weibo 有 store_image，douyin 图文只能下载图片、无独立图文记录）。
功能描述
1. 微信公众号文章批量采集
- 入口A：指定公众号(biz/微信号/主页) → 翻页拉历史文章
（标题、时间、阅读/点赞/在看、正文、封面、配图） - 入口B：指定文章链接/ID → 抓取单篇详情
- 登录：微信扫码(Cookie 复用)
2. 通用图文采集
- 对支持图文的平台，统一抽取「封面+配图+正文」，
落库为独立图文记录，而非仅作为视频附属
技术可行性（草稿，待讨论）
- 公众号数据来源：mp 文章页(需登录态/强反爬) 或
搜狗微信(免登录但限流严重) - 反爬风险高，建议先以「单篇 + 指定号有限页」为范围，
遵守 LICENSE 非商用/限速原则 - 图文落库可复用 AbstractStoreImage / store_image 思路
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Review the existing platform collectors and the AbstractStoreImage/store_image path mentioned in the issue, then compare the mp article-page and Sogou WeChat entry points. The work needs an agreed MVP, supported inputs, storage shape, and acceptance checks before implementation can be considered done.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100