NanmiCoder / NanmiCoder/MediaCrawler

[问题] 运行过程中总是报错

Open
#645 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

question
Dominant language
Python
Stars
65.3k
Forks
12.6k
PR merge metrics
No merged PRs in 30d

Description

⚠️ 提交前确认

  • [✔] 我已经仔细阅读了项目使用过程中的常见问题汇总
  • [✔ ] 我已经搜索并查看了已关闭的issues
  • [✔ ] 我确认这不是由于滑块验证码、Cookie过期、Cookie提取错误、平台风控等常见原因导致的问题

❓ 问题描述

运行日志中总出现类似报错
2025-07-08 17:45:09 MediaCrawler ERROR (core.py:322) - [XiaoHongShuCrawler.get_note_detail_async_task] Get note detail error, note_id: 68568782000000001201628d
2025-07-08 17:45:12 httpx INFO (_client.py:1740) - HTTP Request: POST https://edith.xiaohongshu.com/api/sns/web/v1/feed "HTTP/1.1 200 OK"
爬取的数据一直在81条,也不退出程序,但是日志中一直存在类似报错

🔍 使用场景

  • 目标平台: 小红书
  • 使用功能: 关键词搜索

💻 环境信息

  • 操作系统: win11
  • Python版本: 3.13
  • 是否使用IP代理: 否
  • 是否使用VPN翻墙软件:是
  • 目标平台(抖音/小红书/微博等): 小红书

📋 错误日志

2025-07-08 17:46:41 MediaCrawler ERROR (core.py:322) - [XiaoHongShuCrawler.get_note_detail_async_task] Get note detail error, note_id: 66d43652000000001d039ae5
2025-07-08 17:46:43 httpx INFO (_client.py:1740) - HTTP Request: POST https://edith.xiaohongshu.com/api/sns/web/v1/feed "HTTP/1.1 200 OK"
2025-07-08 17:46:43 MediaCrawler INFO (core.py:307) - [get_note_detail_async_task] Begin get note detail, note_id: 67ac8cc1000000002503eeb8
2025-07-08 17:46:45 httpx INFO (_client.py:1740) - HTTP Request: GET https://www.xiaohongshu.com/discovery/item/67ac8cc1000000002503eeb8 "HTTP/1.1 302 Found"
2025-07-08 17:46:48 httpx INFO (_client.py:1740) - HTTP Request: GET https://www.xiaohongshu.com/discovery/item/67ac8cc1000000002503eeb8 "HTTP/1.1 302 Found"

## 📷 错误截图
<!-- 请提供错误截图 -->

<img width="1660" height="224" alt="Image" src="https://github.com/user-attachments/assets/1f154e4f-af8e-43f0-a55d-3ad9afdd5f30" />

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with core.py:322 and XiaoHongShuCrawler.get_note_detail_async_task, then reproduce the Xiaohongshu keyword-search run using the supplied logs and environment details. Trace the failing note-detail requests and the repeated 302 responses; done means the cause is identified and the crawl no longer remains stuck at 81 items with recurring errors.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.