NanmiCoder / NanmiCoder/MediaCrawler

[BUG] 哔哩哔哩爬虫失败

Open
#834 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
65.3k
Forks
12.6k
No merged PRs in 30d

Description

🔍 问题检查清单

  • 我已经仔细阅读了项目使用过程中的常见问题汇总
  • 我已经搜索并查看了已关闭的issues
  • 我确认这不是由于滑块验证码、Cookie过期、Cookie提取错误、平台风控等常见原因导致的问题

🐛 问题描述

B站爬虫异常已通过以下行为逐个排除是否是平台风控：
1、通过本地Chrome登录相同账号，浏览、互动、创作者空间访问和展开均无问题。
2、更换登录其他账号，使用代码【uv run main.py --platform bili --lt qrcode --type creator】运行，依然报相同异常。
3、同时更换IP和账号，使用代码【uv run main.py --platform bili --lt qrcode --type creator】依然报相同异常。
4、通过其他电脑登录相同账号，浏览、互动、创作者空间访问和展开均无问题。

📝 复现步骤

  1. 登录账号
  2. 使用代码【uv run main.py --platform bili --lt qrcode --type creator】，通过创造者ID获取发布作品信息
  3. 引起异常

💻 运行环境

  • 操作系统:
  • Python版本: 3.11
  • 是否使用IP代理: YES
  • 是否使用VPN翻墙软件：NO
  • 目标平台(抖音/小红书/微博等): 哔哩哔哩

📋 错误日志

2026-02-17 18:20:29 MediaCrawler INFO (core.py:121) - [BilibiliCrawler.start] Parsed creator ID: 3537115461716620 from 3537115461716620
2026-02-17 18:20:34 MediaCrawler ERROR (client.py:76) - [BilibiliClient.request] Failed to decode JSON from response. status_code: 412, response_text: <!DOCTYPE html>
<html lang="zh-cn">

<head>
    <meta http-equiv="Access-Control-Allow-Origin" content="*" />
    <meta http-equiv="Page-Enter" content="blendTrans(Duration=0.5)">
    <meta http-equiv="Page-Exit" content="blendTrans(Duration=0.5)">
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
    <meta name="viewport" content="width=device-width, user-scalable=no, initial-scale=1.0, maximum-scale=1.0, minimum-scale=1.0">
    <meta name="spm_prefix" content="333.937">
    <title>出错啦! - bilibili.com</title>
    <link rel="shortcut icon" href="//static.hdslb.com/images/favicon.ico">
    <script type="text/javascript" src="//s1.hdslb.com/bfs/static/jinkela/long/js/jquery/jquery1.7.2.min.js"></script>
    <style>
        body {
          margin: 0;
          padding: 0;
          width: 100vw;
          height: 100vh;
        }
       .error-container {
          height: 100%;
          width: 100%;
          display: flex;
          justify-content: center;
          align-items: center;
        }
        .captcha-content {
          text-align: center;
        }
        .txt-wrap {
          margin-top: 15px;
          font-size: 12px;
          line-height: 22px;
          color: #999;
          text-align: left;
        }
        .txt-item.err-code {
          font-size: 28px;
          font-weight: 700;
          text-align: center;
          color: #4c93ff;
          margin-top: 16px;
        }

        .captcha-img {
          width: 160px;
          height: 160px;
          margin: auto;
          background: url('https://i0.hdslb.com/bfs/static/jinkela/long/412/%E9%A1%B5%E9%9D%A2%E7%8A%B6%E6%80%81%E5%9B%BE.png') no-repeat center / cover;
        }
        .err-text {
          color: #03a9f4;
        }
        .err-default {
          font-size: 16px;
          margin-top: 10px;
        }
      </style>
</head>

<body>
    <noscript>
          <strong>抱歉！我们需要允许JavaScript才能够正常运行。请打开后重试~</strong>
          <strong>错误：412</strong>
      </noscript>
      <div class="error-container">
        <div class="captcha-content">
          <div class="captcha-img"></div>
          <div class="txt-item err-code">错误号: 412</div>
          <div class="txt-item err-text"></div>
          <div class="txt-item err-default">由于触发哔哩哔哩安全风控策略，该次访问请求被拒绝。
            <div class="txt-item">The request was rejected because of the bilibili security control policy.</div>
          </div>
          <div class="txt-wrap">
            <div class="txt-item datetime_now"></div>
            <div class="txt-item user_url"></div>
            <div class="txt-item user_ip"></div>
            <div class="txt-item user_id"></div>
          </div>
          <div class="check-input">
              <div class="title"></div>
              <div class="box-pic"></div>
              <div class="box"></div>
              <div class="state"></div>
          </div>
      </div>
    </div>
    <script type="text/javascript" charset="utf-8" src="//security.bilibili.com/static/js/sha256.min.js"></script>
    <script type="text/javascript" charset="utf-8" src="//security.bilibili.com/static/js/js.cookie.min.js"></script>
    <script type="text/javascript" charset="utf-8" src="//security.bilibili.com/static/js/412.js"></script>
</body>
</html>
2026-02-17 18:20:34 MediaCrawler INFO (cdp_browser.py:391) - [CDPBrowserManager] Browser connection disconnected
2026-02-17 18:20:34 MediaCrawler INFO (browser_launcher.py:255) - [BrowserLauncher] Closing browser process...
2026-02-17 18:20:39 MediaCrawler WARNING (browser_launcher.py:264) - [BrowserLauncher] Normal termination timeout, using taskkill to force kill
2026-02-17 18:20:42 MediaCrawler INFO (browser_launcher.py:287) - [BrowserLauncher] Browser process closed

📷 错误截图

Image

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the request failure reported at client.py:76 and the creator crawl entry point at core.py:121, then reproduce the command shown in the issue. Review the surrounding browser shutdown entries in cdp_browser.py and browser_launcher.py. Done would require identifying a project-side fix for the Bilibili 412 response, but the issue does not define the expected behavior or a test.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.