DragonflyRobotics / DragonflyRobotics/MAGIST-Algorithm

Google Image Search API Caps at 100 Images Download

Open
#7 1 comment 0 reactions 1 assignee Claimed by @DragonflyRobotics View on GitHub
bug
Dominant language
Python
Stars
5
Forks
0
PR merge metrics
No merged PRs in 30d

Description

**Description**
The Google Image Search API was updated to have a limit of 100 images per query. This means that consecutively downloading, for example, 200 images will fail.

**To Reproduce**
To reproduce the behavior, run the following code:
```python
from MAGIST.Utils.WebScraper.google import GoogleScraper

g = GoogleScraper("config.json")
g.download_raw_img_dataset("cars", 200, "Data/")
```

**Expected behavior**
It should ideally download 200 images and verify their integrity before downloading.

**Screenshots**
![image](https://user-images.githubusercontent.com/85193239/174666564-bfb6e210-d09f-4540-a27e-caaa3415cc73.png)
![image](https://user-images.githubusercontent.com/85193239/174666602-8d2e890b-077d-47a4-8062-4075412a7c35.png)

**Additional context**
I have teamed up with [@arrrlo](https://github.com/arrrlo) who made the `GoogleImageSearch` module for Python with the [issue](https://github.com/arrrlo/Google-Images-Search/issues/149) also posted on [@arrrlo's](https://github.com/arrrlo) [repo](https://github.com/arrrlo/Google-Images-Search).

This requires the module `pyparsing==2.4.2` which is part of a circular dependency issue(#4 ).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.