NanmiCoder / NanmiCoder/MediaCrawler
[问题] IP代理配置后无法使用
Open
Nobody has claimed this yet.
question
- Dominant language
- Python
- Stars
- 65.3k
- Forks
- 12.6k
- PR merge metrics
- No merged PRs in 30d
Description
⚠️ 提交前确认
- 我已经仔细阅读了项目使用过程中的常见问题汇总
- 我已经搜索并查看了已关闭的issues
- 我确认这不是由于滑块验证码、Cookie过期、Cookie提取错误、平台风控等常见原因导致的问题
❓ 问题描述
🔍 使用场景
- 目标平台: (如:小红书/抖音/微博等)
- 使用功能: (如:关键词搜索/用户主页爬取等)
💻 环境信息
- 操作系统:
- Python版本: 2.10
- 是否使用IP代理: 是
- 是否使用VPN翻墙软件:是
- 目标平台(抖音/小红书/微博等): 小红书
📋 错误日志
2025-07-11 16:07:50 MediaCrawler INFO (proxy_ip_pool.py:59) - [ProxyIpPool._is_valid_proxy] testing 58.19.55.17 is it valid
2025-07-11 16:07:55 MediaCrawler INFO (core.py:71) - [XiaoHongShuCrawler] 使用标准模式启动浏览器
2025-07-11 16:07:55 MediaCrawler INFO (core.py:440) - [XiaoHongShuCrawler.launch_browser] Begin create browser context ...
Traceback (most recent call last):
File "D:\MediaCrawler-main\main.py", line 67, in <module>
asyncio.get_event_loop().run_until_complete(main())
File "D:\Anaconda_envs\envs\cralwer\lib\asyncio\base_events.py", line 649, in run_until_complete
return future.result()
File "D:\MediaCrawler-main\main.py", line 57, in main
await crawler.start()
File "D:\MediaCrawler-main\media_platform\xhs\core.py", line 91, in start
await self.context_page.goto(self.index_url)
File "D:\Anaconda_envs\envs\cralwer\lib\site-packages\playwright\async_api\_generated.py", line 8818, in goto
await self._impl_obj.goto(
File "D:\Anaconda_envs\envs\cralwer\lib\site-packages\playwright\_impl\_page.py", line 524, in goto
return await self._main_frame.goto(**locals_to_params(locals()))
File "D:\Anaconda_envs\envs\cralwer\lib\site-packages\playwright\_impl\_frame.py", line 145, in goto
await self._channel.send("goto", locals_to_params(locals()))
File "D:\Anaconda_envs\envs\cralwer\lib\site-packages\playwright\_impl\_connection.py", line 59, in send
return await self._connection.wrap_api_call(
File "D:\Anaconda_envs\envs\cralwer\lib\site-packages\playwright\_impl\_connection.py", line 514, in wrap_api_call
raise rewrite_error(error, f"{parsed_st['apiName']}: {error}") from None
playwright._impl._errors.Error: Page.goto: net::ERR_PROXY_CONNECTION_FAILED at https://www.xiaohongshu.com/
Call log:
navigating to "https://www.xiaohongshu.com/", waiting until "load"
📷 错误截图
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with media_platform/xhs/core.py and proxy_ip_pool.py, then reproduce the command shown in the report and inspect the Playwright Page.goto failure. Verify how the proxy is validated and passed to the browser context; done means the Xiaohongshu page loads successfully with the configured proxy and the failure is covered by a clear diagnostic or test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- playwright, python
- Domain
- backend, networking
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100