hellock / hellock/icrawler

example in readme not work

Open
#136 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
933
Forks
181
PR merge metrics
No merged PRs in 30d

Description

i just test the repo with example code in README.md
```
from icrawler.builtin import GoogleImageCrawler

google_crawler = GoogleImageCrawler(storage={'root_dir': 'your_image_dir'})
google_crawler.crawl(keyword='cat', max_num=100)
```
Following is the results
```
(linebot_env) D:\Project\convenient_script\multi_platform_bot>pip install icrawler
Collecting icrawler
Using cached icrawler-0.6.10-py3-none-any.whl.metadata (6.2 kB)
Requirement already satisfied: beautifulsoup4 in .\linebot_env\lib\site-packages (from icrawler) (4.11.1)
Requirement already satisfied: bs4 in .\linebot_env\lib\site-packages (from icrawler) (0.0.2)
Requirement already satisfied: lxml in .\linebot_env\lib\site-packages (from icrawler) (6.1.0)
Requirement already satisfied: pillow in .\linebot_env\lib\site-packages (from icrawler) (11.3.0)
Requirement already satisfied: pyyaml in .\linebot_env\lib\site-packages (from icrawler) (6.0.3)
Requirement already satisfied: requests in .\linebot_env\lib\site-packages (from icrawler) (2.31.0)
Requirement already satisfied: six in .\linebot_env\lib\site-packages (from icrawler) (1.16.0)
Requirement already satisfied: soupsieve>1.2 in .\linebot_env\lib\site-packages (from beautifulsoup4->icrawler) (2.3.2.post1)
Requirement already satisfied: charset-normalizer<4,>=2 in .\linebot_env\lib\site-packages (from requests->icrawler) (2.1.1)
Requirement already satisfied: idna<4,>=2.5 in .\linebot_env\lib\site-packages (from requests->icrawler) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in .\linebot_env\lib\site-packages (from requests->icrawler) (1.26.12)
Requirement already satisfied: certifi>=2017.4.17 in .\linebot_env\lib\site-packages (from requests->icrawler) (2026.4.22)
Using cached icrawler-0.6.10-py3-none-any.whl (36 kB)
Installing collected packages: icrawler
Successfully installed icrawler-0.6.10

(linebot_env) D:\Project\convenient_script\multi_platform_bot>python test_cf.py
2026-05-11 01:25:11,079 - INFO - icrawler.crawler - start crawling...
2026-05-11 01:25:11,079 - INFO - icrawler.crawler - starting 1 feeder threads...
2026-05-11 01:25:11,079 - INFO - feeder - thread feeder-001 exit
2026-05-11 01:25:11,079 - INFO - icrawler.crawler - starting 1 parser threads...
2026-05-11 01:25:11,080 - INFO - icrawler.crawler - starting 1 downloader threads...
2026-05-11 01:25:11,167 - INFO - parser - parsing result page https://www.google.com/search?q=cat&ijn=0&start=0&tbs=&tbm=isch
Exception in thread parser-001:
Traceback (most recent call last):
File "C:\Users\ZZZ\AppData\Local\Programs\Python\Python39\lib\threading.py", line 980, in _bootstrap_inner
self.run()
File "C:\Users\ZZZ\AppData\Local\Programs\Python\Python39\lib\threading.py", line 917, in run
self._target(*self._args, **self._kwargs)
File "D:\Project\convenient_script\multi_platform_bot\linebot_env\lib\site-packages\icrawler\parser.py", line 93, in worker_exec
for task in self.parse(response, **kwargs):
TypeError: 'NoneType' object is not iterable
2026-05-11 01:25:16,080 - INFO - downloader - no more download task for thread downloader-001
2026-05-11 01:25:16,080 - INFO - downloader - thread downloader-001 exit
2026-05-11 01:25:16,084 - INFO - icrawler.crawler - Crawling task done!

(linebot_env) D:\Project\convenient_script\multi_platform_bot>
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.