NanmiCoder / NanmiCoder/MediaCrawler
[问题] 使用微博关键词搜索的时候直接浏览器崩溃获取不到信息,是不是被微博墙了?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 65.3k
- Forks
- 12.6k
- PR merge metrics
- No merged PRs in 30d
Description
⚠️ 提交前确认
- 我已经仔细阅读了项目使用过程中的常见问题汇总
- 我已经搜索并查看了已关闭的issues
- 我确认这不是由于滑块验证码、Cookie过期、Cookie提取错误、平台风控等常见原因导致的问题
❓ 问题描述
使用微博关键词搜索的时候直接浏览器崩溃获取不到信息,是不是被微博墙了?
🔍 使用场景
- 目标平台: (如:小红书/抖音/微博等)
- 使用功能: (如:关键词搜索/用户主页爬取等)
已删除过浏览器数据并重新登录过
使用微博关键词搜索的时候直接浏览器崩溃获取不到信息,是不是被微博墙了?
💻 环境信息
- 操作系统: win11
- Python版本:
- 是否使用IP代理: 无
- 是否使用VPN翻墙软件:无
- 目标平台(抖音/小红书/微博等): 微博
📋 错误日志
在此粘贴完整的错误日志
2025-05-20 09:36:28 MediaCrawler INFO (core.py:315) - [WeiboCrawler.launch_browser] Begin create browser context ...
2025-05-20 09:36:29 MediaCrawler INFO (core.py:278) - [WeiboCrawler.create_weibo_client] Begin create weibo API client ...
2025-05-20 09:36:29 MediaCrawler INFO (client.py:92) - [WeiboClient.pong] Begin pong weibo...
2025-05-20 09:36:29 MediaCrawler INFO (core.py:110) - [WeiboCrawler.search] Begin search weibo keywords
2025-05-20 09:36:29 MediaCrawler INFO (core.py:117) - [WeiboCrawler.search] Current search keyword: 中美
2025-05-20 09:36:29 MediaCrawler INFO (core.py:124) - [WeiboCrawler.search] search weibo keyword: 中美, page: 1
2025-05-20 09:36:29 MediaCrawler ERROR (client.py:70) - [WeiboClient.request] request GET:https://m.weibo.cn/api/container/getIndex?containerid=100103type%3D1%26q%3D%E4%B8%AD%E7%BE%8E&page_type=searchall&page=1 err, res:{'ok': -100, 'errno': '-100', 'msg': None, 'url': 'https://m.weibo.cn/api/geetest?testType=1&backUrl=https%3A%2F%2Fm.weibo.cn', 'extra': None}
Traceback (most recent call last):
File "E:\MediaCrawler-main\main.py", line 66, in
asyncio.get_event_loop().run_until_complete(main())
File "C:\ProgramData\miniconda3\envs\xhs\lib\asyncio\base_events.py", line 642, in run_until_complete
return future.result()
File "E:\MediaCrawler-main\main.py", line 56, in main
await crawler.start()
File "E:\MediaCrawler-main\media_platform\weibo\core.py", line 94, in start
await self.search()
File "E:\MediaCrawler-main\media_platform\weibo\core.py", line 125, in search
search_res = await self.wb_client.get_note_by_keyword(
File "E:\MediaCrawler-main\media_platform\weibo\client.py", line 131, in get_note_by_keyword
return await self.get(uri, params)
File "E:\MediaCrawler-main\media_platform\weibo\client.py", line 83, in get
return await self.request(method="GET", url=f"{self._host}{final_uri}", headers=headers, **kwargs)
File "E:\MediaCrawler-main\media_platform\weibo\client.py", line 71, in request
raise DataFetchError(data.get("msg", "unknown error"))
media_platform.weibo.exception.DataFetchError: None
📷 错误截图
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in media_platform/weibo/client.py, especially request(), get(), and get_note_by_keyword(), then trace the call from media_platform/weibo/core.py search(). Reproduce the request to the shown m.weibo.cn endpoint and inspect how the geetest response becomes DataFetchError(None). Done should establish whether this is an application-handling bug or an external access restriction and define the resulting behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100