NanmiCoder / NanmiCoder/MediaCrawler
[问题] 运行过程中总是报错
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 65.3k
- Forks
- 12.6k
- PR merge metrics
- No merged PRs in 30d
Description
⚠️ 提交前确认
- [✔] 我已经仔细阅读了项目使用过程中的常见问题汇总
- [✔ ] 我已经搜索并查看了已关闭的issues
- [✔ ] 我确认这不是由于滑块验证码、Cookie过期、Cookie提取错误、平台风控等常见原因导致的问题
❓ 问题描述
运行日志中总出现类似报错
2025-07-08 17:45:09 MediaCrawler ERROR (core.py:322) - [XiaoHongShuCrawler.get_note_detail_async_task] Get note detail error, note_id: 68568782000000001201628d
2025-07-08 17:45:12 httpx INFO (_client.py:1740) - HTTP Request: POST https://edith.xiaohongshu.com/api/sns/web/v1/feed "HTTP/1.1 200 OK"
爬取的数据一直在81条,也不退出程序,但是日志中一直存在类似报错
🔍 使用场景
- 目标平台: 小红书
- 使用功能: 关键词搜索
💻 环境信息
- 操作系统: win11
- Python版本: 3.13
- 是否使用IP代理: 否
- 是否使用VPN翻墙软件:是
- 目标平台(抖音/小红书/微博等): 小红书
📋 错误日志
2025-07-08 17:46:41 MediaCrawler ERROR (core.py:322) - [XiaoHongShuCrawler.get_note_detail_async_task] Get note detail error, note_id: 66d43652000000001d039ae5
2025-07-08 17:46:43 httpx INFO (_client.py:1740) - HTTP Request: POST https://edith.xiaohongshu.com/api/sns/web/v1/feed "HTTP/1.1 200 OK"
2025-07-08 17:46:43 MediaCrawler INFO (core.py:307) - [get_note_detail_async_task] Begin get note detail, note_id: 67ac8cc1000000002503eeb8
2025-07-08 17:46:45 httpx INFO (_client.py:1740) - HTTP Request: GET https://www.xiaohongshu.com/discovery/item/67ac8cc1000000002503eeb8 "HTTP/1.1 302 Found"
2025-07-08 17:46:48 httpx INFO (_client.py:1740) - HTTP Request: GET https://www.xiaohongshu.com/discovery/item/67ac8cc1000000002503eeb8 "HTTP/1.1 302 Found"
## 📷 错误截图
<!-- 请提供错误截图 -->
<img width="1660" height="224" alt="Image" src="https://github.com/user-attachments/assets/1f154e4f-af8e-43f0-a55d-3ad9afdd5f30" />
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with core.py:322 and XiaoHongShuCrawler.get_note_detail_async_task, then reproduce the Xiaohongshu keyword-search run using the supplied logs and environment details. Trace the failing note-detail requests and the repeated 302 responses; done means the cause is identified and the crawl no longer remains stuck at 81 items with recurring errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100