hellock / hellock/icrawler

Download more than 1000 images using BingImageCrawler

Open
#70 4 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
Python
Stars
933
Forks
181
PR merge metrics
No merged PRs in 30d

Description

Hey! I am trying to download 3000 images for each keyword using BingImageCrawler but I am getting cut off at < 1000 images per keyword. The documentation says `To crawl more than 1000 images with a single keyword, we can specify different date ranges`. This works easily for GoogleImageCrawler because it supports a tuple of date ranges but BingImageCrawler doesn't (only supports “pastday”, “pastweek”, “pastmonth”, “pastyear”).

How can I set the configs such that I can get more than 1000 images per keyword? Also, if I need to do multiple request for each keyword, how can I set the date filter so that I get non-overlapping images? (Using any two of “pastday”, “pastweek”, “pastmonth”, “pastyear” will always result in overlapping images).

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.