hellock / hellock/icrawler

TypeError: 'NoneType' object is not iterable

Open
#107 23 comments 0 reactions 0 assignees View on GitHub
bug help wanted
Dominant language
Python
Stars
933
Forks
181
PR merge metrics
No merged PRs in 30d

Description

For me the GoogleImageCrawler of icrawler doesn't work anymore. I updated the user agent in `crawler.py` since that seemed to work in the past, but no luck here. I tried it both on python 3.8 and 3.9 (apple silicon, but shouldn't matter). Again, it worked in the past (like 3-6 months ago).

Even the simple example

```
from icrawler.builtin import GoogleImageCrawler
searchterm = 'ANY SEARCHTERM'
google_crawler = GoogleImageCrawler(storage={'root_dir': 'test'})
google_crawler.crawl(keyword=searchterm, max_num=1)
```

gives
```
...lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File ".../python3.8/site-packages/icrawler-0.6.6-py3.8.egg/icrawler/parser.py", line 104, in worker_exec
for task in self.parse(response, **kwargs):
TypeError: 'NoneType' object is not iterable
```

Does anyone know how to fix this, or have the same issue in July 2022?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.