hellock / hellock/icrawler

GoogleImageCrawler: TypeError: 'NoneType' object is not iterable

Open
#96 15 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
933
Forks
181
PR merge metrics
No merged PRs in 30d

Description

Hi there, I just tried out your library, but unfortunately, I get an error:
```
2021-03-06 13:56:54,609 - INFO - icrawler.crawler - start crawling...
2021-03-06 13:56:54,609 - INFO - icrawler.crawler - starting 1 feeder threads...
2021-03-06 13:56:54,610 - INFO - feeder - thread feeder-001 exit
2021-03-06 13:56:54,611 - INFO - icrawler.crawler - starting 1 parser threads...
2021-03-06 13:56:54,612 - INFO - icrawler.crawler - starting 1 downloader threads...
2021-03-06 13:56:55,160 - INFO - parser - parsing result page https://www.google.com/search?q=cat&ijn=0&start=0&tbs=&tbm=isch
Exception in thread parser-001:
Traceback (most recent call last):
File "C:\Users\WILLI\AppData\Local\Programs\Python\Python38\lib\threading.py", line 932, in _bootstrap_inner
self.run()
File "C:\Users\WILLI\AppData\Local\Programs\Python\Python38\lib\threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "D:\Projekte\Foo\source\backend_prototyp\venv\lib\site-packages\icrawler\parser.py", line 104, in worker_exec
for task in self.parse(response, **kwargs):
TypeError: 'NoneType' object is not iterable
2021-03-06 13:56:59,615 - INFO - downloader - no more download task for thread downloader-001
2021-03-06 13:56:59,615 - INFO - downloader - thread downloader-001 exit
2021-03-06 13:56:59,616 - INFO - icrawler.crawler - Crawling task done!

Process finished with exit code 0
```

But I just use your example code:
```
from icrawler.builtin import GoogleImageCrawler

google_crawler = GoogleImageCrawler(storage={'root_dir': 'D:'})
google_crawler.crawl(keyword='cat', max_num=100)
```

How can I fix the problem?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.