dandi / dandi/dandi-api-webshots
chrome driver started to die off??
- Dominant language
- No language data
- Stars
- 0
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
not sure what has changed in the past two days, we started to get emails with crashes like
```
+ xvfb-run python tools/make_webshots.py -i dandi
2022-06-17T21:30:17-0400 [WARNING ] MainProcess[1724596]: dandi: A newer version (0.40.1) of dandi/dandi-cli is available. You are using 0.38.0
2022-06-17T21:30:23-0400 [INFO ] Process-1[1725490]: make_webshots: Logging in ...
tools/make_webshots.py:131: DeprecationWarning: find_element_by_* commands are deprecated. Please use find_element() instead
username_field = self.driver.find_element_by_id("login_field")
tools/make_webshots.py:132: DeprecationWarning: find_element_by_* commands are deprecated. Please use find_element() instead
password_field = self.driver.find_element_by_id("password")
tools/make_webshots.py:136: DeprecationWarning: find_elements_by_tag_name is deprecated. Please use find_elements(by=By.TAG_NAME, value=name) instead
self.driver.find_elements_by_tag_name("form")[0].submit()
tools/make_webshots.py:143: DeprecationWarning: find_element_by_xpath is deprecated. Please use find_element(by=By.XPATH, value=xpath) instead
self.driver.find_element_by_xpath(
tools/make_webshots.py:154: DeprecationWarning: find_elements_by_class_name is deprecated. Please use find_elements(by=By.CLASS_NAME, value=name) instead
or self.driver.find_elements_by_class_name("v-avatar")
2022-06-17T21:30:29-0400 [INFO ] Process-1[1725490]: make_webshots: 000003 landing
2022-06-17T21:30:35-0400 [INFO ] Process-1[1725490]: make_webshots: 000003 edit-metadata
Process Process-1:
Traceback (most recent call last):
File "/home/dandi/miniconda3/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/home/dandi/miniconda3/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "tools/make_webshots.py", line 410, in snapshot_pipe
t = ws.process_dandiset_page(ds, urlsuf, page, wait_cls, pbar_cls, act)
File "tools/make_webshots.py", line 241, in process_dandiset_page
self.driver.get(f"{self.gui_url}/dandiset/{ds}")
File "/mnt/backup/dandi/dandi-api-webshots/venv/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 437, in get
self.execute(Command.GET, {'url': url})
File "/mnt/backup/dandi/dandi-api-webshots/venv/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 425, in execute
self.error_handler.check_response(response)
File "/mnt/backup/dandi/dandi-api-webshots/venv/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 247, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: net::ERR_SSL_PROTOCOL_ERROR
(Session info: headless chrome=100.0.4896.60)
Stacktrace:
#0 0x55d01a0c7af3
#1 0x55d019dd88b6
#2 0x55d019dd676a
#3 0x55d019dc2ae7
#4 0x55d019dc39be
#5 0x55d019dc2de4
#6 0x55d019dc213b
#7 0x55d019dc1042
#8 0x55d019dc14ac
#9 0x55d019ddaba4
#10 0x55d019e47374
#11 0x55d019e34762
#12 0x55d019e4702b
#13 0x55d019e34623
#14 0x55d019e0427b
#15 0x55d019e05aae
#16 0x55d01a0f391d
#17 0x55d01a105ea3
#18 0x55d01a105b99
#19 0x55d01a106462
#20 0x55d01a0f4e3b
#21 0x55d01a1066e1
#22 0x55d01a0e95c9
#23 0x55d01a11d1d5
#24 0x55d01a11d373
#25 0x55d01a136f41
#26 0x7fab75120ea7
2022-06-17T21:31:07-0400 [WARNING ] MainProcess[1724596]: make_webshots: Subprocess exited while processing ('000003', 'edit-metadata'); restarting
2022-06-17T21:31:07-0400 [INFO ] Process-2[1726124]: make_webshots: Logging in ...
tools/make_webshots.py:131: DeprecationWarning: find_element_by_* commands are deprecated. Please use find_element() instead
```
and then finally die entirely with
```
2022-06-17T21:43:09-0400 [WARNING ] MainProcess[1724596]: make_webshots: Subprocess exited while processing ('000008', 'view-data'); restarting
2022-06-17T21:43:10-0400 [INFO ] Process-18[1730267]: make_webshots: Logging in ...
tools/make_webshots.py:131: DeprecationWarning: find_element_by_* commands are deprecated. Please use find_element() instead
username_field = self.driver.find_element_by_id("login_field")
tools/make_webshots.py:132: DeprecationWarning: find_element_by_* commands are deprecated. Please use find_element() instead
password_field = self.driver.find_element_by_id("password")
tools/make_webshots.py:136: DeprecationWarning: find_elements_by_tag_name is deprecated. Please use find_elements(by=By.TAG_NAME, value=name) instead
self.driver.find_elements_by_tag_name("form")[0].submit()
tools/make_webshots.py:143: DeprecationWarning: find_element_by_xpath is deprecated. Please use find_element(by=By.XPATH, value=xpath) instead
self.driver.find_element_by_xpath(
2022-06-17T21:43:13-0400 [INFO ] Process-18[1730267]: make_webshots: Cleaning up 7 child processes
Process Process-18:
Traceback (most recent call last):
File "/home/dandi/miniconda3/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/home/dandi/miniconda3/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "tools/make_webshots.py", line 401, in snapshot_pipe
with Webshotter(gui_url, headless, login) as ws:
File "tools/make_webshots.py", line 89, in __init__
self.set_driver()
File "tools/make_webshots.py", line 111, in set_driver
self.login(os.environ["DANDI_USERNAME"], os.environ["DANDI_PASSWORD"])
File "tools/make_webshots.py", line 149, in login
raise RateLimitError("GitHub secondary rate limit exceeded")
RateLimitError: GitHub secondary rate limit exceeded
Traceback (most recent call last):
File "tools/make_webshots.py", line 552, in
main()
File "/mnt/backup/dandi/dandi-api-webshots/venv/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/mnt/backup/dandi/dandi-api-webshots/venv/lib/python3.8/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/mnt/backup/dandi/dandi-api-webshots/venv/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/mnt/backup/dandi/dandi-api-webshots/venv/lib/python3.8/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "tools/make_webshots.py", line 477, in main
stats.append(ff(ds, page))
File "tools/make_webshots.py", line 331, in __call__
raise RuntimeError(
RuntimeError: Subprocess encountered unrecoverable error: GitHub secondary rate limit exceeded
```
that is why we don't have webshots updated here for two days now. Happened on June 16 and 17.
Any ideas @jwodder ?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.