NanmiCoder / NanmiCoder/MediaCrawler

Add a lightweight preflight check for cookies/proxy before starting a crawl

Open
#960 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
65.3k
Forks
12.6k
PR merge metrics
No merged PRs in 30d

Description

I've been using MediaCrawler with saved cookies for a few platforms, and the most common failure mode is not obvious until the crawl has already started: the session expired, the proxy is blocked, or the cookie file is for the wrong account/platform.

A small diagnostic command would save a lot of trial and error. For example, python main.py --check-session --platform xhs could load the current login state, make one low-impact authenticated request, and print whether the session is usable. It could also mention the likely cause if it fails, such as expired cookie, proxy error, or unexpected response.

This doesn't need to auto-fix login or refresh tokens. Even a simple pass/fail status for each configured platform would make setup and scheduled runs much less painful, especially when multiple platforms are involved.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at main.py and trace how the --platform option selects a platform and loads its current login state. Review the existing authenticated requests and proxy handling before defining a low-impact check for each configured platform. Done means --check-session reports a clear pass or failure reason without starting a crawl or changing login state.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
authentication, cli, networking
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.