NanmiCoder / NanmiCoder/MediaCrawler

第一次把数据存储到mysql或csv时有数据,再次爬取时换了新的数据库或把原来的csv移到别的地方或换成json格式后,就没有数据了(数据库是空的,data目录是空的)

Open
#622 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
65.3k
Forks
12.6k
PR merge metrics
No merged PRs in 30d

Description

没有把原来的csv移到别的地方,再次爬取时,文件内容不会变,也没有新的文件出现

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing a second crawl after switching between MySQL, CSV, and JSON, including moving the existing CSV, then trace the storage-selection and output paths. Compare the first and subsequent runs to determine why the database and data directory remain empty. Done means the selected storage receives the expected records on subsequent crawls.

Written by the indexing model from the issue text.

Assessment

Tech stack
json, mysql, python
Domain
data, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.