NanmiCoder / NanmiCoder/MediaCrawler
[BUG]
Open
Nobody has claimed this yet.
bug
- Dominant language
- Python
- Stars
- 65.3k
- Forks
- 12.6k
- PR merge metrics
- No merged PRs in 30d
Description
🔍 问题检查清单
- 我已经仔细阅读了项目使用过程中的常见问题汇总
- 我已经搜索并查看了已关闭的issues
- 我确认这不是由于滑块验证码、Cookie过期、Cookie提取错误、平台风控等常见原因导致的问题
🐛 问题描述
在正常配置完环境后运行出现TargetClosedError的情况,仅在Windows上出现,wsl中测试是正常的。
📝 复现步骤
- 在Windows上正常配置环境
- 运行程序
- 报出错误
💻 运行环境
- 操作系统: Windows11
- Python版本: 3.9.6
- 是否使用IP代理: 否
- 是否使用VPN翻墙软件:无论是否使用都不行
- 目标平台(抖音/小红书/微博等): 只测试了xhs和dy,应该是共性问题
📋 错误日志
(crawler) PS D:\Code\Work\Projects\MediaCrawler> uv run main.py --platform dy --lt qrcode --type search --save_data_option csv --keywords "职高"
2025-07-21 20:32:30 MediaCrawler INFO (core.py:72) - [DouYinCrawler] 使用标准模式启动浏览器
2025-07-21 20:32:39 MediaCrawler INFO (login.py:116) - [DouYinLogin.login_by_qrcode] Begin login douyin by qrcode...
2025-07-21 20:32:48 MediaCrawler INFO (login.py:70) - [DouYinLogin.begin] login finished then check login state ...
2025-07-21 20:33:21 MediaCrawler INFO (login.py:79) - [DouYinLogin.begin] Login successful then wait for 5 seconds redirect ...
2025-07-21 20:33:26 MediaCrawler INFO (core.py:113) - [DouYinCrawler.search] Begin search douyin keywords
2025-07-21 20:33:26 MediaCrawler INFO (core.py:120) - [DouYinCrawler.search] Current keyword: 职高
2025-07-21 20:33:26 MediaCrawler INFO (core.py:128) - [DouYinCrawler.search] Skip 0
2025-07-21 20:33:26 MediaCrawler INFO (core.py:132) - [DouYinCrawler.search] search douyin keyword: 职高, page: 1
Traceback (most recent call last):
File "D:\Code\Work\Projects\MediaCrawler\main.py", line 79, in <module>
cleanup()
File "D:\Code\Work\Projects\MediaCrawler\main.py", line 70, in cleanup
asyncio.run(crawler.close())
File "C:\Users\31933\anaconda3\envs\crawler\lib\asyncio\runners.py", line 44, in run
return loop.run_until_complete(main)
File "C:\Users\31933\anaconda3\envs\crawler\lib\asyncio\base_events.py", line 642, in run_until_complete
return future.result()
File "D:\Code\Work\Projects\MediaCrawler\media_platform\douyin\core.py", line 385, in close
await self.browser_context.close()
File "D:\Code\Work\Projects\MediaCrawler\.venv\lib\site-packages\playwright\async_api\_generated.py", line 13164, in close
return mapping.from_maybe_impl(await self._impl_obj.close(reason=reason))
File "D:\Code\Work\Projects\MediaCrawler\.venv\lib\site-packages\playwright\_impl\_browser_context.py", line 552, in close
await self._channel.send("close", {"reason": reason})
File "D:\Code\Work\Projects\MediaCrawler\.venv\lib\site-packages\playwright\_impl\_connection.py", line 59, in send
return await self._connection.wrap_api_call(
File "D:\Code\Work\Projects\MediaCrawler\.venv\lib\site-packages\playwright\_impl\_connection.py", line 514, in wrap_api_call
raise rewrite_error(error, f"{parsed_st['apiName']}: {error}") from None
playwright._impl._errors.TargetClosedError: BrowserContext.close: Target page, context or browser has been closed
2025-07-21 20:33:36 asyncio ERROR (base_events.py:1738) - Task was destroyed but it is pending!
task: <Task pending name='Task-1' coro=<ExpiringLocalCache._start_clear_cron() running at D:\Code\Work\Projects\MediaCrawler\cache\local_cache.py:119> wait_for=<Future cancelled>>
2025-07-21 20:33:36 asyncio ERROR (base_events.py:1738) - Task was destroyed but it is pending!
task: <Task pending name='Task-2' coro=<ExpiringLocalCache._start_clear_cron() running at D:\Code\Work\Projects\MediaCrawler\cache\local_cache.py:119> wait_for=<Future cancelled>>
📷 错误截图
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with main.py cleanup and media_platform/douyin/core.py close, where the reported BrowserContext.close error occurs. Then inspect cache/local_cache.py around the pending ExpiringLocalCache tasks and reproduce the command on Windows. Done means the Windows run no longer reports TargetClosedError or destroyed pending tasks during cleanup.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- playwright, python
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100