jupyterhub / jupyterhub/systemdspawner
A user servers's systemd service in a failed state fails to start up properly again
- Dominant language
- Python
- Stars
- 100
- Forks
- 44
- PR merge metrics
- No merged PRs in 30d
Description
I'm having problems where if I leave a hub running for an extended period of time (weeks or longer) it becomes slow to respond and sometimes unable to start a user's server, for example:
```
May 27 03:07:25 littlehub python3[1910]: [I 2019-05-27 03:07:25.408 JupyterHub proxy:301] Checking routes
May 27 03:07:33 littlehub python3[1910]: [W 2019-05-27 03:07:33.210 JupyterHub base:714] User lukeedwards is slow to start (timeout=10)
May 27 03:07:33 littlehub python3[1910]: [I 2019-05-27 03:07:33.212 JupyterHub log:158] 202 POST /hub/api/users/lukeedwards/server (lachlan@106.69.254.220) 10064.93ms
May 27 03:08:02 littlehub python3[1910]: [I 2019-05-27 03:08:02.138 JupyterHub log:158] 200 GET /hub/admin (lachlan@106.69.254.220) 49.57ms
May 27 03:08:23 littlehub python3[1910]: [W 2019-05-27 03:08:23.294 JupyterHub user:471] lukeedwards's server failed to start in 60 seconds, giving up
May 27 03:08:23 littlehub python3[1910]: [E 2019-05-27 03:08:23.380 JupyterHub gen:593] Exception in Future .finish_user_spawn() done, defined at /opt/tljh/hub/lib/python3.6/site-packages/jupyterhub/handlers/base.py:619> exception=TimeoutError('Timeout',)> after timeout
May 27 03:08:23 littlehub python3[1910]: Traceback (most recent call last):
May 27 03:08:23 littlehub python3[1910]: File "/opt/tljh/hub/lib/python3.6/site-packages/tornado/gen.py", line 589, in error_callback
May 27 03:08:23 littlehub python3[1910]: future.result()
May 27 03:08:23 littlehub python3[1910]: File "/opt/tljh/hub/lib/python3.6/site-packages/jupyterhub/handlers/base.py", line 626, in finish_user_spawn
May 27 03:08:23 littlehub python3[1910]: await spawn_future
May 27 03:08:23 littlehub python3[1910]: File "/opt/tljh/hub/lib/python3.6/site-packages/jupyterhub/user.py", line 489, in spawn
May 27 03:08:23 littlehub python3[1910]: raise e
May 27 03:08:23 littlehub python3[1910]: File "/opt/tljh/hub/lib/python3.6/site-packages/jupyterhub/user.py", line 409, in spawn
May 27 03:08:23 littlehub python3[1910]: url = await gen.with_timeout(timedelta(seconds=spawner.start_timeout), f)
May 27 03:08:23 littlehub python3[1910]: tornado.util.TimeoutError: Timeout
```
I've tried reloading the hub, and restarting the VM (Ubuntu 18.04 on Openstack) with no success.
Contributor guide
Assessment
This issue has not been assessed yet.