NanmiCoder / NanmiCoder/MediaCrawler

[BUG]

Open
#801 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
65.3k
Forks
12.6k
PR merge metrics
No merged PRs in 30d

Description

🔍 问题检查清单

  • 我已经仔细阅读了项目使用过程中的常见问题汇总
  • 我已经搜索并查看了已关闭的issues
  • 我确认这不是由于滑块验证码、Cookie过期、Cookie提取错误、平台风控等常见原因导致的问题

🐛 问题描述

用的cookie 关键词爬图 瓦屋山森林公园 爬356张图 大概50多个帖子 就会出现这样的报错 连续好几次

📝 复现步骤

💻 运行环境

  • 操作系统: win10
  • Python版本: 3.11.14
  • 是否使用IP代理: 否
  • 是否使用VPN翻墙软件:否
  • 目标平台(抖音/小红书/微博等): 微博

📋 错误日志

2025-12-25 02:26:06 MediaCrawler INFO (cdp_browser.py:391) - [CDPBrowserManager] 浏览器连接已断开
2025-12-25 02:26:06 MediaCrawler INFO (browser_launcher.py:255) - [BrowserLauncher] 正在关闭浏览器进程...
2025-12-25 02:26:06 MediaCrawler INFO (browser_launcher.py:285) - [BrowserLauncher] 浏览器进程已关闭
Traceback (most recent call last):
  File "E:\Users\Desktop\Code\python\MediaCrawler\main.py", line 146, in <module>
    run(main, async_cleanup, cleanup_timeout_seconds=15.0, on_first_interrupt=_force_stop)
  File "E:\Users\Desktop\Code\python\MediaCrawler\tools\app_runner.py", line 109, in run
    asyncio.run(_runner())
  File "C:\Users\93244\AppData\Roaming\uv\python\cpython-3.11.14-windows-x86_64-none\Lib\asyncio\runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "C:\Users\93244\AppData\Roaming\uv\python\cpython-3.11.14-windows-x86_64-none\Lib\asyncio\runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\93244\AppData\Roaming\uv\python\cpython-3.11.14-windows-x86_64-none\Lib\asyncio\base_events.py", line 654, in run_until_complete        
    return future.result()
           ^^^^^^^^^^^^^^^
  File "E:\Users\Desktop\Code\python\MediaCrawler\tools\app_runner.py", line 96, in _runner
    await app_main()
  File "E:\Users\Desktop\Code\python\MediaCrawler\main.py", line 99, in main
    await crawler.start()
  File "E:\Users\Desktop\Code\python\MediaCrawler\media_platform\weibo\core.py", line 125, in start
    await self.search()
  File "E:\Users\Desktop\Code\python\MediaCrawler\media_platform\weibo\core.py", line 179, in search
    await self.get_note_images(mblog)
  File "E:\Users\Desktop\Code\python\MediaCrawler\media_platform\weibo\core.py", line 283, in get_note_images
    url = pic.get("url")
          ^^^^^^^
AttributeError: 'str' object has no attribute 'get'

📷 错误截图

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in media_platform/weibo/core.py at get_note_images(), especially line 283 where the traceback shows pic.get("url") failing. Reproduce a Weibo keyword crawl for 瓦屋山森林公园 and inspect the image data passed into this method; done means the crawl no longer stops with AttributeError and image collection still works.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.