NanmiCoder / NanmiCoder/MediaCrawler

快手评论区爬取不完全

Open
#843 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
65.3k
Forks
12.6k
PR merge metrics
No merged PRs in 30d

Description

🔍 问题检查清单

  • 我已经仔细阅读了项目使用过程中的常见问题汇总
  • 我已经搜索并查看了已关闭的issues
  • 我确认这不是由于滑块验证码、Cookie过期、Cookie提取错误、平台风控等常见原因导致的问题

🐛 问题描述

📝 复现步骤

快手爬虫,评论区爬取不完全,会出现EST API V2 error

  1. PLATFORM = "ks" # Platform, xhs | dy | ks | bili | wb | tieba | zhihu
  2. Control the number of crawled videos/posts

CRAWLER_MAX_NOTES_COUNT = 500
3. # Control the number of crawled first-level comments (single video/post)
CRAWLER_MAX_COMMENTS_COUNT_SINGLENOTES = 1000

💻 运行环境

  • 操作系统:
  • Python版本: 3.10
  • 是否使用IP代理: 是
  • 是否使用VPN翻墙软件:是
  • 目标平台(抖音/小红书/微博等): 快手

📋 错误日志

在此粘贴错误日志

2026-03-09 22:05:51 MediaCrawler ERROR (core.py:284) - [KuaishouCrawler.get_comments] get video_id: 3xe2e9fheibixs6 comment error: REST API V2 error: {'result': 2, 'error_msg': None, 'request_id': '773065155769196843'}
2026

📷 错误截图

Image

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at core.py:284 and the KuaishouCrawler.get_comments entry point named in the log. Reproduce with the provided Python 3.10 settings and inspect the complete REST API V2 response and surrounding crawler flow. Done means the configured comments are retrieved without the reported error, with the missing-comment behavior explained by a regression test or documented cause.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.