NanmiCoder / NanmiCoder/MediaCrawler

[问题] 使用微博关键词搜索的时候直接浏览器崩溃获取不到信息,是不是被微博墙了?

Open
#604 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

question
Dominant language
Python
Stars
65.3k
Forks
12.6k
PR merge metrics
No merged PRs in 30d

Description

⚠️ 提交前确认

  • 我已经仔细阅读了项目使用过程中的常见问题汇总
  • 我已经搜索并查看了已关闭的issues
  • 我确认这不是由于滑块验证码、Cookie过期、Cookie提取错误、平台风控等常见原因导致的问题

❓ 问题描述

使用微博关键词搜索的时候直接浏览器崩溃获取不到信息,是不是被微博墙了?

🔍 使用场景

  • 目标平台: (如:小红书/抖音/微博等)
  • 使用功能: (如:关键词搜索/用户主页爬取等)
    已删除过浏览器数据并重新登录过
    使用微博关键词搜索的时候直接浏览器崩溃获取不到信息,是不是被微博墙了?

💻 环境信息

  • 操作系统: win11
  • Python版本:
  • 是否使用IP代理: 无
  • 是否使用VPN翻墙软件:无
  • 目标平台(抖音/小红书/微博等): 微博

📋 错误日志

在此粘贴完整的错误日志

2025-05-20 09:36:28 MediaCrawler INFO (core.py:315) - [WeiboCrawler.launch_browser] Begin create browser context ...
2025-05-20 09:36:29 MediaCrawler INFO (core.py:278) - [WeiboCrawler.create_weibo_client] Begin create weibo API client ...
2025-05-20 09:36:29 MediaCrawler INFO (client.py:92) - [WeiboClient.pong] Begin pong weibo...
2025-05-20 09:36:29 MediaCrawler INFO (core.py:110) - [WeiboCrawler.search] Begin search weibo keywords
2025-05-20 09:36:29 MediaCrawler INFO (core.py:117) - [WeiboCrawler.search] Current search keyword: 中美
2025-05-20 09:36:29 MediaCrawler INFO (core.py:124) - [WeiboCrawler.search] search weibo keyword: 中美, page: 1
2025-05-20 09:36:29 MediaCrawler ERROR (client.py:70) - [WeiboClient.request] request GET:https://m.weibo.cn/api/container/getIndex?containerid=100103type%3D1%26q%3D%E4%B8%AD%E7%BE%8E&page_type=searchall&page=1 err, res:{'ok': -100, 'errno': '-100', 'msg': None, 'url': 'https://m.weibo.cn/api/geetest?testType=1&backUrl=https%3A%2F%2Fm.weibo.cn', 'extra': None}
Traceback (most recent call last):
File "E:\MediaCrawler-main\main.py", line 66, in
asyncio.get_event_loop().run_until_complete(main())
File "C:\ProgramData\miniconda3\envs\xhs\lib\asyncio\base_events.py", line 642, in run_until_complete
return future.result()
File "E:\MediaCrawler-main\main.py", line 56, in main
await crawler.start()
File "E:\MediaCrawler-main\media_platform\weibo\core.py", line 94, in start
await self.search()
File "E:\MediaCrawler-main\media_platform\weibo\core.py", line 125, in search
search_res = await self.wb_client.get_note_by_keyword(
File "E:\MediaCrawler-main\media_platform\weibo\client.py", line 131, in get_note_by_keyword
return await self.get(uri, params)
File "E:\MediaCrawler-main\media_platform\weibo\client.py", line 83, in get
return await self.request(method="GET", url=f"{self._host}{final_uri}", headers=headers, **kwargs)
File "E:\MediaCrawler-main\media_platform\weibo\client.py", line 71, in request
raise DataFetchError(data.get("msg", "unknown error"))
media_platform.weibo.exception.DataFetchError: None

📷 错误截图

Image

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in media_platform/weibo/client.py, especially request(), get(), and get_note_by_keyword(), then trace the call from media_platform/weibo/core.py search(). Reproduce the request to the shown m.weibo.cn endpoint and inspect how the geetest response becomes DataFetchError(None). Done should establish whether this is an application-handling bug or an external access restriction and define the resulting behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.