hellock / hellock/icrawler

GoogleImageCrawler returns no results (TypeError: 'NoneType' object is not iterable)

Open
#135 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
933
Forks
181
PR merge metrics
No merged PRs in 30d

Description

Hi,

I’m trying to use GoogleImageCrawler with the following script:

```
from icrawler.builtin import GoogleImageCrawler
from icrawler import ImageDownloader
import os

# Custom image downloader without format filtering
class CustomDownloader(ImageDownloader):
def _filter(self, task):
return super()._filter(task) # No extension/type filtering now

def download_images(plant_name, category, num_images=300):
save_dir = os.path.join('dataset', category.replace(' ', '_'), plant_name.replace(' ', '_'))
os.makedirs(save_dir, exist_ok=True)

crawler = GoogleImageCrawler(
downloader_cls=CustomDownloader,
storage={'root_dir': save_dir}
)

crawler.crawl(
keyword=f"{plant_name} plant images",
max_num=num_images,
min_size=(512, 512),
file_idx_offset=0
)

if __name__ == '__main__':
plant_name = input("Enter plant name (e.g., 'Loropetalum 'Plum''): ").strip()
category = input("Enter category (e.g., 'Small or Mass Planting Shrub'): ").strip()

download_images(plant_name, category)
print("✅ Done! Check your dataset folder.")

```
**Error / Logs:**
```
2025-08-21 14:47:15,712 - INFO - parser - parsing result page https://www.google.com/search?q=coconut+palm+plant+images&ijn=0&start=0&tbs=&tbm=isch
Exception in thread parser-001:
Traceback (most recent call last):
...
File ".../icrawler/parser.py", line 93, in worker_exec
for task in self.parse(response, **kwargs):
TypeError: 'NoneType' object is not iterable
```
**Environment:**
- OS: Ubuntu 20.04
- Python: 3.10
- icrawler version: (please confirm with pip show icrawler)

**Notes:**
- Baidu and Bing crawlers work fine.
- Only GoogleImageCrawler fails with the above error.
- Looks like Google changed its HTML/response format.

Is there any fix or workaround planned for Google support?
Thanks!

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.