404 Trying to connect to single-user notebook page

(Note: I've removed my username, domain name, and indentifiable IP addresses just to be safe.)
### Bug description
After the SLURM process is started, the user is directed to a 404 page.

#### Expected behaviour
- On /hub/spawn, I click the spawn button.
- I see a loading bar while batchspawner waits for the process to start.
- I'm brought to /user/aerobinsoniv@contoso.com/tree and I see a list of my notebooks.

#### Actual behaviour
- After the loading bar, I'm brought to a 404 page.

### How to reproduce
This happens every time I try to use Jupyterhub. Probably system-specific, because it's unavoidable.

#### Setup

- OS:
CentOS 7

- Version(s):
Jupyterhub 1.4.1, Python 3.9.4, Conda 4.10.1, Batchspawner 1.1.0

#### Other relevant info:

- I have access to a working equivalent of this setup (using an older version of Jupyterhub). Most of the GET requests that 404 on the new version to `/user/aerobinsoniv@contoso.com/[path]` on the working version, but the new version lacks the `/user/aerobinsoniv@contoso.com` part.

- The working version sets cookies in the way described here: https://jupyterhub.readthedocs.io/en/0.7.2/howitworks.html. The new version sets them differently.

- A lot of 302s occur in response to the get requests, promptng my browser to try the specified location, which 404s.

- I have attachted a HAR file showing the interaction between my browser (Firefox 89.0) and the site up to when the problem occurs.

#### Logs

Full environment

```
XDG_SESSION_ID=182
HOSTNAME=jupyterhub-test
SELINUX_ROLE_REQUESTED=
__LMOD_REF_COUNT_MODULEPATH=/opt/ohpc/pub/modulefiles:1
TERM=xterm-256color
SHELL=/bin/bash
HISTSIZE=1000
SSH_CLIENT=aaa.aaa.aaa.aaa 63602 22
CONDA_SHLVL=1
CONDA_PROMPT_MODIFIER=(jupyterenv)
SELINUX_USE_CURRENT_RANGE=
LMOD_PKG=/opt/ohpc/admin/lmod/lmod
OLDPWD=/home/iv
LMOD_VERSION=7.8.1
SSH_TTY=/dev/pts/0
USER=iv
LS_COLORS=rs=0:di=38;5;27:ln=38;5;51:mh=44;38;5;15:pi=40;38;5;11:so=38;5;13:do=38;5;5:bd=48;5;232;38;5;11:cd=48;5;232;38;5;3:or=48;5;232;38;5;9:mi=05;48;5;232;38;5;15:su=48;5;196;38;5;15:sg=48;5;11;38;5;16:ca=48;5;196;38;5;226:tw=48;5;10;38;5;16:ow=48;5;10;38;5;21:st=48;5;21;38;5;15:ex=38;5;34:*.tar=38;5;9:*.tgz=38;5;9:*.arc=38;5;9:*.arj=38;5;9:*.taz=38;5;9:*.lha=38;5;9:*.lz4=38;5;9:*.lzh=38;5;9:*.lzma=38;5;9:*.tlz=38;5;9:*.txz=38;5;9:*.tzo=38;5;9:*.t7z=38;5;9:*.zip=38;5;9:*.z=38;5;9:*.Z=38;5;9:*.dz=38;5;9:*.gz=38;5;9:*.lrz=38;5;9:*.lz=38;5;9:*.lzo=38;5;9:*.xz=38;5;9:*.bz2=38;5;9:*.bz=38;5;9:*.tbz=38;5;9:*.tbz2=38;5;9:*.tz=38;5;9:*.deb=38;5;9:*.rpm=38;5;9:*.jar=38;5;9:*.war=38;5;9:*.ear=38;5;9:*.sar=38;5;9:*.rar=38;5;9:*.alz=38;5;9:*.ace=38;5;9:*.zoo=38;5;9:*.cpio=38;5;9:*.7z=38;5;9:*.rz=38;5;9:*.cab=38;5;9:*.jpg=38;5;13:*.jpeg=38;5;13:*.gif=38;5;13:*.bmp=38;5;13:*.pbm=38;5;13:*.pgm=38;5;13:*.ppm=38;5;13:*.tga=38;5;13:*.xbm=38;5;13:*.xpm=38;5;13:*.tif=38;5;13:*.tiff=38;5;13:*.png=38;5;13:*.svg=38;5;13:*.svgz=38;5;13:*.mng=38;5;13:*.pcx=38;5;13:*.mov=38;5;13:*.mpg=38;5;13:*.mpeg=38;5;13:*.m2v=38;5;13:*.mkv=38;5;13:*.webm=38;5;13:*.ogm=38;5;13:*.mp4=38;5;13:*.m4v=38;5;13:*.mp4v=38;5;13:*.vob=38;5;13:*.qt=38;5;13:*.nuv=38;5;13:*.wmv=38;5;13:*.asf=38;5;13:*.rm=38;5;13:*.rmvb=38;5;13:*.flc=38;5;13:*.avi=38;5;13:*.fli=38;5;13:*.flv=38;5;13:*.gl=38;5;13:*.dl=38;5;13:*.xcf=38;5;13:*.xwd=38;5;13:*.yuv=38;5;13:*.cgm=38;5;13:*.emf=38;5;13:*.axv=38;5;13:*.anx=38;5;13:*.ogv=38;5;13:*.ogx=38;5;13:*.aac=38;5;45:*.au=38;5;45:*.flac=38;5;45:*.mid=38;5;45:*.midi=38;5;45:*.mka=38;5;45:*.mp3=38;5;45:*.mpc=38;5;45:*.ogg=38;5;45:*.ra=38;5;45:*.wav=38;5;45:*.axa=38;5;45:*.oga=38;5;45:*.spx=38;5;45:*.xspf=38;5;45:
CONDA_EXE=/usr/local/anaconda3/bin/conda
_CE_CONDA=
LMOD_PREPEND_BLOCK=normal
_ModuleTable001_=X01vZHVsZVRhYmxlXz17WyJNVHZlcnNpb24iXT0zLFsiY19yZWJ1aWxkVGltZSJdPWZhbHNlLFsiY19zaG9ydFRpbWUiXT1mYWxzZSxkZXB0aFQ9e30sZmFtaWx5PXt9LG1UPXt9LG1wYXRoQT17Ii9vcHQvb2hwYy9wdWIvbW9kdWxlZmlsZXMiLH0sWyJzeXN0ZW1CYXNlTVBBVEgiXT0iL29wdC9vaHBjL3B1Yi9tb2R1bGVmaWxlcyIsfQ==
MAIL=/var/spool/mail/iv
PATH=/home/iv/.conda/envs/jupyterenv/bin:/usr/local/anaconda3/condabin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/iv/.local/bin:/home/iv/bin
CONDA_PREFIX=/home/iv/.conda/envs/jupyterenv
LMOD_SETTARG_CMD=:
PWD=/home/iv/jupyterhub
LANG=en_US.UTF-8
MODULEPATH=/opt/ohpc/pub/modulefiles
_ModuleTable_Sz_=1
SELINUX_LEVEL_REQUESTED=
LMOD_CMD=/opt/ohpc/admin/lmod/lmod/libexec/lmod
_CE_M=
HISTCONTROL=ignoredups
SHLVL=1
HOME=/home/iv
BASH_ENV=/opt/ohpc/admin/lmod/lmod/init/bash
CONDA_PYTHON_EXE=/usr/local/anaconda3/bin/python
LOGNAME=iv
SSH_CONNECTION=aaa.aaa.aaa.aaa 63602 aaa.aaa.aaa.aaa 22
MODULESHOME=/opt/ohpc/admin/lmod/lmod
CONDA_DEFAULT_ENV=jupyterenv
LMOD_SETTARG_FULL_SUPPORT=no
LESSOPEN=||/usr/bin/lesspipe.sh %s
LMOD_FULL_SETTARG_SUPPORT=no
XDG_RUNTIME_DIR=/run/user/1002
LMOD_DIR=/opt/ohpc/admin/lmod/lmod/libexec
LMOD_COLORIZE=no
BASH_FUNC_module()=() { eval $($LMOD_CMD bash "$@") && eval $(${LMOD_SETTARG_CMD:-:} -s sh)
}
BASH_FUNC_ml()=() { eval $($LMOD_DIR/ml_cmd "$@")
}
_=/usr/bin/env
```

conda list output

```
(jupyterenv) [iv@jupyterhub-test jupyterhub]$ conda list
# packages in environment at /home/iv/.conda/envs/jupyterenv:
#
# Name Version Build Channel
_libgcc_mutex 0.1 main
alembic 1.6.4 pyhd8ed1ab_0 conda-forge
anyio 2.2.0 py39h06a4308_1
argon2-cffi 20.1.0 py39h27cfd23_1
async_generator 1.10 pyhd3eb1b0_0
attrs 21.2.0 pyhd3eb1b0_0
babel 2.9.1 pyhd3eb1b0_0
backcall 0.2.0 pyhd3eb1b0_0
batchspawner 1.1.0 pypi_0 pypi
bleach 3.3.0 pyhd3eb1b0_0
blinker 1.4 py_1 conda-forge
brotlipy 0.7.0 py39h27cfd23_1003
ca-certificates 2020.12.5 ha878542_0 conda-forge
certifi 2020.12.5 py39hf3d152e_1 conda-forge
certipy 0.1.3 py_0 conda-forge
cffi 1.14.5 py39h261ae71_0
chardet 4.0.0 py39h06a4308_1003
configurable-http-proxy 4.3.2 node12_hac6e442_0 conda-forge
cryptography 3.4.7 py39hd23ed53_0
decorator 5.0.9 pyhd3eb1b0_0
defusedxml 0.7.1 pyhd3eb1b0_0
elementpath 2.2.2 pypi_0 pypi
entrypoints 0.3 py39h06a4308_0
icu 68.1 h58526e2_0 conda-forge
idna 2.10 pyhd3eb1b0_0
importlib-metadata 3.10.0 py39h06a4308_0
importlib-resources 5.1.2 pypi_0 pypi
importlib_metadata 3.10.0 hd3eb1b0_0
iniconfig 1.1.1 pypi_0 pypi
ipykernel 5.3.4 py39hb070fc8_0
ipython 7.22.0 py39hb070fc8_0
ipython_genutils 0.2.0 pyhd3eb1b0_1
jedi 0.17.2 py39h06a4308_1
jinja2 3.0.0 pyhd3eb1b0_0
json5 0.9.5 py_0
jsonschema 3.2.0 py_2
jupyter-packaging 0.7.12 pyhd3eb1b0_0
jupyter-saml2authenticator 0.1.0.dev0 pypi_0 pypi
jupyter_client 6.1.12 pyhd3eb1b0_0
jupyter_core 4.7.1 py39h06a4308_0
jupyter_server 1.4.1 py39h06a4308_0
jupyter_telemetry 0.1.0 pyhd8ed1ab_1 conda-forge
jupyterhub 1.4.1 py39hf3d152e_0 conda-forge
jupyterhub-base 1.4.1 py39hf3d152e_0 conda-forge
jupyterlab 3.0.14 pyhd8ed1ab_0 conda-forge
jupyterlab_pygments 0.1.2 py_0
jupyterlab_server 2.4.0 pyhd3eb1b0_0
krb5 1.18.2 h173b8e3_0
ld_impl_linux-64 2.33.1 h53a641e_7
libcurl 7.71.1 h20c2e04_1
libedit 3.1.20191231 he28a2e2_2 conda-forge
libffi 3.3 he6710b0_2
libgcc-ng 9.1.0 hdf63c60_0
libsodium 1.0.18 h7b6447c_0
libssh2 1.9.0 hab1572f_5 conda-forge
libstdcxx-ng 9.1.0 hdf63c60_0
libuv 1.40.0 h7b6447c_0
mako 1.1.4 pyh44b312d_0 conda-forge
markupsafe 2.0.1 py39h27cfd23_0
mistune 0.8.4 py39h27cfd23_1000
nbclassic 0.2.6 pyhd3eb1b0_0
nbclient 0.5.3 pyhd3eb1b0_0
nbconvert 6.0.7 py39h06a4308_0
nbformat 5.1.3 pyhd3eb1b0_0
ncurses 6.2 he6710b0_1
nest-asyncio 1.5.1 pyhd3eb1b0_0
nodejs 12.19.0 hfa01f41_2 conda-forge
notebook 6.4.0 py39h06a4308_0
oauthlib 3.0.1 py_0 conda-forge
openssl 1.1.1k h27cfd23_0
packaging 20.9 pyhd3eb1b0_0
pamela 1.0.0 py_0 conda-forge
pandoc 2.12 h06a4308_0
pandocfilters 1.4.3 py39h06a4308_1
parso 0.7.0 py_0
pexpect 4.8.0 pyhd3eb1b0_3
pickleshare 0.7.5 pyhd3eb1b0_1003
pip 21.1.1 py39h06a4308_0
pluggy 0.13.1 pypi_0 pypi
prometheus_client 0.10.1 pyhd3eb1b0_0
prompt-toolkit 3.0.17 pyh06a4308_0
ptyprocess 0.7.0 pyhd3eb1b0_2
py 1.10.0 pypi_0 pypi
pycparser 2.20 py_2
pycurl 7.43.0.6 py39h8cd828d_0 conda-forge
pygments 2.9.0 pyhd3eb1b0_0
pyjwt 2.1.0 pyhd8ed1ab_0 conda-forge
pyopenssl 20.0.1 pyhd3eb1b0_1
pyparsing 2.4.7 pyhd3eb1b0_0
pyrsistent 0.17.3 py39h27cfd23_0
pysaml2 6.5.1 pypi_0 pypi
pysocks 1.7.1 py39h06a4308_0
pytest 6.2.4 pypi_0 pypi
python 3.9.4 hdb3f193_0
python-dateutil 2.8.1 pyhd3eb1b0_0
python-editor 1.0.4 py_0 conda-forge
python-json-logger 2.0.1 pyh9f0ad1d_0 conda-forge
python_abi 3.9 1_cp39 conda-forge
pytz 2021.1 pyhd3eb1b0_0
pyzmq 20.0.0 py39h2531618_1
readline 8.1 h27cfd23_0
requests 2.25.1 pyhd3eb1b0_0
ruamel.yaml 0.16.12 py39h27cfd23_1
ruamel.yaml.clib 0.2.2 py39hbd71b63_1 conda-forge
send2trash 1.5.0 pyhd3eb1b0_1
setuptools 52.0.0 py39h06a4308_0
six 1.15.0 py39h06a4308_0
sniffio 1.2.0 py39h06a4308_1
sqlalchemy 1.3.23 py39h27cfd23_0
sqlite 3.35.4 hdfb4753_0
terminado 0.9.4 py39h06a4308_0
testpath 0.4.4 pyhd3eb1b0_0
tk 8.6.10 hbc83047_0
toml 0.10.2 pypi_0 pypi
tornado 6.1 py39h27cfd23_0
traitlets 5.0.5 pyhd3eb1b0_0
tzdata 2020f h52ac0ba_0
urllib3 1.26.4 pyhd3eb1b0_0
wcwidth 0.2.5 py_0
webencodings 0.5.1 py39h06a4308_1
wheel 0.36.2 pyhd3eb1b0_0
wrapspawner 1.0.0 pypi_0 pypi
xmlschema 1.6.2 pypi_0 pypi
xz 5.2.5 h7b6447c_0
zeromq 4.3.4 h2531618_0
zipp 3.4.1 pyhd3eb1b0_0
zlib 1.2.11 h7b6447c_3
```

Configuration

```python
c.JupyterHub.bind_url = 'https://[jupyterhub-test.contoso.com]:8000/'
c.JupyterHub.cookie_secret_file = 'jupyterhub_cookie_secret'
c.JupyterHub.hub_ip = 'jupyterhub-test.contoso.com'
c.JupyterHub.ip = ''
c.JupyterHub.port = 8000
import batchspawner.api
c = get_config()
c.JupyterHub.spawner_class = 'wrapspawner.ProfilesSpawner'
c.Spawner.http_timeout = 300
c.Spawner.start_timeout = 300
c.BatchSpawnerBase.req_host = 'jupyterhub-test.contoso.com'
c.BatchSpawnerBase.req_nprocs = '2'
c.BatchSpawnerBase.req_memory = '2gb'
c.BatchSpawnerBase.req_runtime = '6:00:00'
c.TorqueSpawner.state_exechost_exp = r'in-\1.jupyterhub-test.contoso.com'
c.ProfilesSpawner.profiles = [
('2 cores/2 GB/6 hours', '2c2gb', 'batchspawner.SlurmSpawner',
dict(req_nprocs='2', req_queue='normal', req_runtime='6:00:00', req_memory='2gb')),
('2 cores/8 GB/2 hours', '2c8gb', 'batchspawner.SlurmSpawner',
dict(req_nprocs='2', req_queue='normal', req_runtime='2:00:00', req_memory='8gb'))
]
c.JupyterHub.ssl_cert = '/opt/jupyterhub/jupyterhub-test.contoso.com.cer'
c.JupyterHub.ssl_key = '/opt/jupyterhub/jupyterhub-test.contoso.com.key'
from jupyter_saml2authenticator import Saml2Authenticator
c.JupyterHub.authenticator_class = Saml2Authenticator
c.Saml2Authenticator.saml2_metadata_url = *Prefer not to share*
c.Saml2Authenticator.saml2_entity_id = 'https://jupyterhub-test.contoso.com/saml2_auth/ent'
c.Saml2Authenticator.saml2_attribute_username = 'emailAddress'
c.Saml2Authenticator.login_service = "Contoso Single Sign-on"
c.Saml2Authenticator.saml2_strip_username = False
c.Saml2Authenticator.saml2_login_url = r'/saml2_auth/login'
c.Saml2Authenticator.saml2_acs_url = r'/saml2_auth/acs'
```

Logs

This is the log from the compute node that the single-user notebook app runs on (errors at the end are from pressing ctrl+c):
```
/usr/local/anaconda3/bin/jupyterhub-singleuser
[I 2021-06-12 11:24:16.326 SingleUserNotebookApp notebookapp:1593] Authentication of /metrics is OFF, since other authentication is disabled.
[I 2021-06-12 11:24:17.826 SingleUserNotebookApp handlers:78] Loading IPython parallel extension
[I 2021-06-12 11:24:20.378 LabApp] JupyterLab extension loaded from /usr/local/anaconda3/lib/python3.9/site-packages/jupyterlab
[I 2021-06-12 11:24:20.379 LabApp] JupyterLab application directory is /usr/local/anaconda3/share/jupyter/lab
Patching auth into jupyter_server.base.handlers.JupyterHandler(jupyter_server.base.handlers.AuthenticatedHandler) -> JupyterHandler(jupyterhub.singleuser.mixins.HubAuthenticatedHandler, jupyter_server.base.handlers.AuthenticatedHandler)
[I 2021-06-12 11:24:20.392 SingleUserNotebookApp mixins:576] Starting jupyterhub-singleuser server version 1.4.1
[I 2021-06-12 11:24:20.399 SingleUserNotebookApp log:189] 302 GET /user/aerobinsoniv@contoso.com/ -> /user/aerobinsoniv@contoso.com (@aaa.aaa.aaa.aaa) 3.08ms
[I 2021-06-12 11:24:20.409 SingleUserNotebookApp notebookapp:2302] Serving notebooks from local directory: /home/arobinson3@admin.contoso.com
[I 2021-06-12 11:24:20.410 SingleUserNotebookApp notebookapp:2302] Jupyter Notebook 6.3.0 is running at:
[I 2021-06-12 11:24:20.410 SingleUserNotebookApp notebookapp:2302] http://jupyterhub-c5:34582/
[I 2021-06-12 11:24:20.410 SingleUserNotebookApp notebookapp:2303] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[I 2021-06-12 11:24:20.442 SingleUserNotebookApp mixins:556] Updating Hub with activity every 300 seconds
[I 2021-06-12 11:24:20.767 SingleUserNotebookApp log:189] 302 GET /user/aerobinsoniv@contoso.com/ -> /user/aerobinsoniv@contoso.com (@aaa.aaa.aaa.aaa) 1.52ms
[W 2021-06-12 11:24:20.992 SingleUserNotebookApp log:189] 404 GET /user/aerobinsoniv@contoso.com (@aaa.aaa.aaa.aaa) 102.75ms
[I 2021-06-12 12:14:13.269 SingleUserNotebookApp log:189] 302 GET /user/aerobinsoniv@contoso.com/ -> /user/aerobinsoniv@contoso.com (@aaa.aaa.aaa.aaa) 2.12ms
[W 2021-06-12 12:14:13.458 SingleUserNotebookApp log:189] 404 GET /user/aerobinsoniv@contoso.com (@aaa.aaa.aaa.aaa) 8.65ms
[W 2021-06-12 12:14:38.003 SingleUserNotebookApp log:189] 404 GET /user/aerobinsoniv@contoso.com (@aaa.aaa.aaa.aaa) 4.26ms
srun: Job step aborted: Waiting up to 32 seconds for job step to finish.
slurmstepd: error: *** JOB 187 ON jupyterhub-c5 CANCELLED AT 2021-06-12T14:37:50 ***
slurmstepd: error: *** STEP 187.0 ON jupyterhub-c5 CANCELLED AT 2021-06-12T14:37:50 ***
[C 2021-06-12 14:37:50.741 SingleUserNotebookApp notebookapp:1978] received signal 15, stopping
[I 2021-06-12 14:37:51.022 SingleUserNotebookApp notebookapp:2145] Shutting down 0 kernels
[I 2021-06-12 14:37:51.022 SingleUserNotebookApp notebookapp:2160] Shutting down 0 terminals
SIGTERM received
jupyterhub-singleuser ended gracefully
```
This is the output from running the `jupyterhub` command on the login node:
```
[I 2021-06-12 17:17:18.610 JupyterHub app:2463] Running JupyterHub version 1.4.1
[I 2021-06-12 17:17:18.623 JupyterHub app:2493] Using Authenticator: jupyter_saml2authenticator.auth.Saml2Authenticator
[I 2021-06-12 17:17:18.623 JupyterHub app:2493] Using Spawner: wrapspawner.wrapspawner.ProfilesSpawner
[I 2021-06-12 17:17:18.623 JupyterHub app:2493] Using Proxy: jupyterhub.proxy.ConfigurableHTTPProxy-1.4.1
[I 2021-06-12 17:17:18.967 JupyterHub app:1534] Loading cookie_secret from /home/iv/jupyterhub/jupyterhub_cookie_secret
[I 2021-06-12 17:17:19.326 JupyterHub proxy:497] Generating new CONFIGPROXY_AUTH_TOKEN
[W 2021-06-12 17:17:19.328 JupyterHub app:1808] No admin users, admin interface will be unavailable.
[W 2021-06-12 17:17:19.328 JupyterHub app:1809] Add any administrative users to `c.Authenticator.admin_users` in config.
[I 2021-06-12 17:17:19.328 JupyterHub app:1838] Not using allowed_users. Any authenticated user will be allowed.
[I 2021-06-12 17:17:19.451 JupyterHub app:2530] Initialized 0 spawners in 0.006 seconds
[I 2021-06-12 17:17:19.461 JupyterHub proxy:703] Starting proxy @ https://jupyterhub-test.contoso.com:8000/
17:17:25.092 [ConfigProxy] info: Proxying https://jupyterhub-test.contoso.com:8000 to (no default)
17:17:25.187 [ConfigProxy] info: Proxy API at http://127.0.0.1:8001/api/routes
17:17:25.679 [ConfigProxy] info: 200 GET /api/routes
[I 2021-06-12 17:17:25.712 JupyterHub app:2778] Hub API listening on http://jupyterhub-test.contoso.com:8081/hub/
17:17:25.714 [ConfigProxy] info: 200 GET /api/routes
[I 2021-06-12 17:17:25.714 JupyterHub proxy:347] Checking routes
[I 2021-06-12 17:17:25.714 JupyterHub proxy:432] Adding route for Hub: / => http://jupyterhub-test.contoso.com:8081
17:17:25.717 [ConfigProxy] info: Adding route / -> http://jupyterhub-test.contoso.com:8081
17:17:25.718 [ConfigProxy] info: Route added / -> http://jupyterhub-test.contoso.com:8081
17:17:25.719 [ConfigProxy] info: 201 POST /api/routes/
[I 2021-06-12 17:17:25.720 JupyterHub app:2853] JupyterHub is now running at https://jupyterhub-test.contoso.com:8000/
[I 2021-06-12 17:17:56.716 JupyterHub log:189] 302 GET / -> /hub/ (@aaa.aaa.aaa.aaa) 43.69ms
[I 2021-06-12 17:17:56.951 JupyterHub log:189] 302 GET /hub/ -> /hub/spawn (aerobinsoniv@contoso.com@aaa.aaa.aaa.aaa) 122.05ms
[I 2021-06-12 17:17:57.202 JupyterHub log:189] 200 GET /hub/spawn (aerobinsoniv@contoso.com@aaa.aaa.aaa.aaa) 140.49ms
[I 2021-06-12 17:17:58.929 JupyterHub provider:574] Creating oauth client jupyterhub-user-aerobinsoniv%40contoso.com
[I 2021-06-12 17:17:59.148 JupyterHub batchspawner:262] Spawner submitting job using sudo -E -u aerobinsoniv@contoso.com sbatch --parsable
[I 2021-06-12 17:17:59.148 JupyterHub batchspawner:263] Spawner submitted script:
#!/bin/bash
#SBATCH --output=/home/aerobinsoniv@admin.contoso.com/jupyterhub_slurmspawner_%j.log
#SBATCH --job-name=spawner-jupyterhub
#SBATCH --chdir=/home/aerobinsoniv@admin.contoso.com
#SBATCH --export=PATH,CONDA_DEFAULT_ENV,LANG,JUPYTERHUB_API_TOKEN,JPY_API_TOKEN,JUPYTERHUB_CLIENT_ID,JUPYTERHUB_HOST,JUPYTERHUB_OAUTH_CALLBACK_URL,JUPYTERHUB_USER,JUPYTERHUB_SERVER_NAME,JUPYTERHUB_API_URL,JUPYTERHUB_ACTIVITY_URL,JUPYTERHUB_BASE_URL,USER,HOME,SHELL
#SBATCH --get-user-env=L
#SBATCH --time=6:00:00
#SBATCH --mem=2gb
#SBATCH --cpus-per-task=2

set -euo pipefail

trap 'echo SIGTERM received' TERM

which jupyterhub-singleuser
srun batchspawner-singleuser jupyterhub-singleuser --ip=0.0.0.0
echo "jupyterhub-singleuser ended gracefully"

[I 2021-06-12 17:17:59.419 JupyterHub batchspawner:266] Job submitted. cmd: sudo -E -u aerobinsoniv@contoso.com sbatch --parsable output: 188
[I 2021-06-12 17:17:59.874 JupyterHub log:189] 302 POST /hub/spawn -> /hub/spawn-pending/aerobinsoniv@contoso.com (aerobinsoniv@contoso.com@aaa.aaa.aaa.aaa) 1009.44ms
[I 2021-06-12 17:18:00.049 JupyterHub pages:402] aerobinsoniv@contoso.com is pending spawn
[I 2021-06-12 17:18:00.061 JupyterHub log:189] 200 GET /hub/spawn-pending/aerobinsoniv@contoso.com (aerobinsoniv@contoso.com@aaa.aaa.aaa.aaa) 32.47ms
[I 2021-06-12 17:18:03.978 JupyterHub api:23] key is port, value 41166
[I 2021-06-12 17:18:03.982 JupyterHub api:24] hasattr(spawner, key) is True
[I 2021-06-12 17:18:03.986 JupyterHub log:189] 201 POST /hub/api/batchspawner (aerobinsoniv@contoso.com@aaa.aaa.aaa.aaa) 40.37ms
[I 2021-06-12 17:18:04.078 JupyterHub batchspawner:420] Notebook server job 188 started at jupyterhub-c5:41166
[I 2021-06-12 17:18:08.636 JupyterHub base:909] User aerobinsoniv@contoso.com took 9.766 seconds to start
[I 2021-06-12 17:18:08.639 JupyterHub proxy:285] Adding user aerobinsoniv@contoso.com to proxy /user/aerobinsoniv@contoso.com/ => http://jupyterhub-c5:41166
[I 2021-06-12 17:18:08.643 JupyterHub log:189] 200 GET /hub/api (@aaa.aaa.aaa.aaa) 1.39ms
17:18:08.668 [ConfigProxy] info: Adding route /user/aerobinsoniv@contoso.com -> http://jupyterhub-c5:41166
17:18:08.669 [ConfigProxy] info: Route added /user/aerobinsoniv@contoso.com -> http://jupyterhub-c5:41166
17:18:08.670 [ConfigProxy] info: 201 POST /api/routes/user/aerobinsoniv@contoso.com
[I 2021-06-12 17:18:08.718 JupyterHub log:189] 200 POST /hub/api/users/aerobinsoniv@contoso.com/activity (aerobinsoniv@contoso.com@aaa.aaa.aaa.aaa) 47.25ms
[I 2021-06-12 17:18:08.719 JupyterHub users:677] Server aerobinsoniv@contoso.com is ready
[I 2021-06-12 17:18:08.720 JupyterHub log:189] 200 GET /hub/api/users/aerobinsoniv@contoso.com/server/progress (aerobinsoniv@contoso.com@aaa.aaa.aaa.aaa) 8398.65ms
[I 2021-06-12 17:18:08.880 JupyterHub log:189] 302 GET /hub/spawn-pending/aerobinsoniv@contoso.com -> /user/aerobinsoniv@contoso.com/ (aerobinsoniv@contoso.com@aaa.aaa.aaa.aaa) 13.14ms
[I 2021-06-12 17:18:09.415 JupyterHub log:189] 302 GET /static/components/jquery-ui/themes/smoothness/jquery-ui.min.css?v=fb45616eef2c454960f91fcd2a04efeda84cfacccf0c5d741ba2793dc1dbd6d3ab01aaae6485222945774c7d7a9a2e9fb87e0d8ef1ea96893aa6906147a371bb -> /hub/static/components/jquery-ui/themes/smoothness/jquery-ui.min.css?v=fb45616eef2c454960f91fcd2a04efeda84cfacccf0c5d741ba2793dc1dbd6d3ab01aaae6485222945774c7d7a9a2e9fb87e0d8ef1ea96893aa6906147a371bb (@aaa.aaa.aaa.aaa) 1.53ms
[I 2021-06-12 17:18:09.418 JupyterHub log:189] 302 GET /static/style/style.min.css?v=56dfd556850eb17b7998c6828467598a322b41593edc758739c66cb2c3fea98f23d0dd8bf8b9b0f5d67bb976a50e4c34f789fe640cbb440fa089e1bf5ec170bd -> /hub/static/style/style.min.css?v=56dfd556850eb17b7998c6828467598a322b41593edc758739c66cb2c3fea98f23d0dd8bf8b9b0f5d67bb976a50e4c34f789fe640cbb440fa089e1bf5ec170bd (@aaa.aaa.aaa.aaa) 0.93ms
[I 2021-06-12 17:18:09.422 JupyterHub log:189] 302 GET /static/components/jquery-typeahead/dist/jquery.typeahead.min.css?v=5edf53bf6bb9c3b1ddafd8594825a7e2ed621f19423e569c985162742f63911c09eba2c529f8fb47aebf27fafdfe287d563347f58c1126b278189a18871b6a9a -> /hub/static/components/jquery-typeahead/dist/jquery.typeahead.min.css?v=5edf53bf6bb9c3b1ddafd8594825a7e2ed621f19423e569c985162742f63911c09eba2c529f8fb47aebf27fafdfe287d563347f58c1126b278189a18871b6a9a (@aaa.aaa.aaa.aaa) 0.95ms
[I 2021-06-12 17:18:09.431 JupyterHub log:189] 302 GET /custom/custom.css -> /hub/custom/custom.css (@aaa.aaa.aaa.aaa) 1.63ms
[I 2021-06-12 17:18:09.432 JupyterHub log:189] 302 GET /static/components/es6-promise/promise.min.js?v=bea335d74136a63ae1b5130f5ac9a50c6256a5f435e6e09fef599491a84d834a8b0f011ca3eaaca3b4ab6a2da2d3e1191567a2f171e60da1d10e5b9d52f84184 -> /hub/static/components/es6-promise/promise.min.js?v=bea335d74136a63ae1b5130f5ac9a50c6256a5f435e6e09fef599491a84d834a8b0f011ca3eaaca3b4ab6a2da2d3e1191567a2f171e60da1d10e5b9d52f84184 (@aaa.aaa.aaa.aaa) 1.63ms
[I 2021-06-12 17:18:09.435 JupyterHub log:189] 302 GET /static/components/react/react.production.min.js?v=9a0aaf84a316c8bedd6c2ff7d5b5e0a13f8f84ec02442346cba0b842c6c81a6bf6176e64f3675c2ebf357cb5bb048e0b527bd39377c95681d22468da3d5de735 -> /hub/static/components/react/react.production.min.js?v=9a0aaf84a316c8bedd6c2ff7d5b5e0a13f8f84ec02442346cba0b842c6c81a6bf6176e64f3675c2ebf357cb5bb048e0b527bd39377c95681d22468da3d5de735 (@aaa.aaa.aaa.aaa) 0.93ms
[I 2021-06-12 17:18:09.531 JupyterHub log:189] 302 GET /static/components/react/react-dom.production.min.js?v=6fc58c1c4736868ff84f57bd8b85f2bdb985993a9392718f3b4af4bfa10fb4efba2b4ddd68644bd2a8daf0619a3844944c9c43f8528364a1aa6fc01ec1b8ae84 -> /hub/static/components/react/react-dom.production.min.js?v=6fc58c1c4736868ff84f57bd8b85f2bdb985993a9392718f3b4af4bfa10fb4efba2b4ddd68644bd2a8daf0619a3844944c9c43f8528364a1aa6fc01ec1b8ae84 (@aaa.aaa.aaa.aaa) 0.99ms
[I 2021-06-12 17:18:09.545 JupyterHub log:189] 302 GET /static/components/create-react-class/index.js?v=894ad57246e682b4cfbe7cd5e408dcd6b38d06af4de4f3425991e2676fdc2ef1732cbd19903104198878ae77de12a1996de3e7da3a467fb226bdda8f4618faec -> /hub/static/components/create-react-class/index.js?v=894ad57246e682b4cfbe7cd5e408dcd6b38d06af4de4f3425991e2676fdc2ef1732cbd19903104198878ae77de12a1996de3e7da3a467fb226bdda8f4618faec (@aaa.aaa.aaa.aaa) 2.38ms
[I 2021-06-12 17:18:09.546 JupyterHub log:189] 302 GET /static/components/requirejs/require.js?v=d37b48bb2137faa0ab98157e240c084dd5b1b5e74911723aa1d1f04c928c2a03dedf922d049e4815f7e5a369faa2e6b6a1000aae958b7953b5cc60411154f593 -> /hub/static/components/requirejs/require.js?v=d37b48bb2137faa0ab98157e240c084dd5b1b5e74911723aa1d1f04c928c2a03dedf922d049e4815f7e5a369faa2e6b6a1000aae958b7953b5cc60411154f593 (@aaa.aaa.aaa.aaa) 2.45ms
[W 2021-06-12 17:18:09.548 JupyterHub log:189] 404 GET /hub/static/components/jquery-ui/themes/smoothness/jquery-ui.min.css?v=fb45616eef2c454960f91fcd2a04efeda84cfacccf0c5d741ba2793dc1dbd6d3ab01aaae6485222945774c7d7a9a2e9fb87e0d8ef1ea96893aa6906147a371bb (@aaa.aaa.aaa.aaa) 3.09ms
[W 2021-06-12 17:18:09.637 JupyterHub log:189] 404 GET /hub/static/style/style.min.css?v=56dfd556850eb17b7998c6828467598a322b41593edc758739c66cb2c3fea98f23d0dd8bf8b9b0f5d67bb976a50e4c34f789fe640cbb440fa089e1bf5ec170bd (@aaa.aaa.aaa.aaa) 1.63ms
[W 2021-06-12 17:18:09.640 JupyterHub log:189] 404 GET /hub/static/components/jquery-typeahead/dist/jquery.typeahead.min.css?v=5edf53bf6bb9c3b1ddafd8594825a7e2ed621f19423e569c985162742f63911c09eba2c529f8fb47aebf27fafdfe287d563347f58c1126b278189a18871b6a9a (@aaa.aaa.aaa.aaa) 1.39ms
[W 2021-06-12 17:18:09.729 JupyterHub log:189] 404 GET /hub/custom/custom.css (aerobinsoniv@contoso.com@aaa.aaa.aaa.aaa) 57.21ms
[W 2021-06-12 17:18:09.733 JupyterHub log:189] 404 GET /hub/static/components/react/react.production.min.js?v=9a0aaf84a316c8bedd6c2ff7d5b5e0a13f8f84ec02442346cba0b842c6c81a6bf6176e64f3675c2ebf357cb5bb048e0b527bd39377c95681d22468da3d5de735 (@aaa.aaa.aaa.aaa) 2.69ms
[W 2021-06-12 17:18:09.734 JupyterHub log:189] 404 GET /hub/static/components/es6-promise/promise.min.js?v=bea335d74136a63ae1b5130f5ac9a50c6256a5f435e6e09fef599491a84d834a8b0f011ca3eaaca3b4ab6a2da2d3e1191567a2f171e60da1d10e5b9d52f84184 (@aaa.aaa.aaa.aaa) 3.16ms
[W 2021-06-12 17:18:09.735 JupyterHub log:189] 404 GET /hub/static/components/react/react-dom.production.min.js?v=6fc58c1c4736868ff84f57bd8b85f2bdb985993a9392718f3b4af4bfa10fb4efba2b4ddd68644bd2a8daf0619a3844944c9c43f8528364a1aa6fc01ec1b8ae84 (@aaa.aaa.aaa.aaa) 3.33ms
[W 2021-06-12 17:18:09.779 JupyterHub log:189] 404 GET /hub/static/components/create-react-class/index.js?v=894ad57246e682b4cfbe7cd5e408dcd6b38d06af4de4f3425991e2676fdc2ef1732cbd19903104198878ae77de12a1996de3e7da3a467fb226bdda8f4618faec (@aaa.aaa.aaa.aaa) 1.23ms
[I 2021-06-12 17:18:09.854 JupyterHub log:189] 302 GET /static/components/react/react.production.min.js?v=9a0aaf84a316c8bedd6c2ff7d5b5e0a13f8f84ec02442346cba0b842c6c81a6bf6176e64f3675c2ebf357cb5bb048e0b527bd39377c95681d22468da3d5de735 -> /hub/static/components/react/react.production.min.js?v=9a0aaf84a316c8bedd6c2ff7d5b5e0a13f8f84ec02442346cba0b842c6c81a6bf6176e64f3675c2ebf357cb5bb048e0b527bd39377c95681d22468da3d5de735 (@aaa.aaa.aaa.aaa) 1.19ms
[W 2021-06-12 17:18:09.961 JupyterHub log:189] 404 GET /hub/static/components/react/react.production.min.js?v=9a0aaf84a316c8bedd6c2ff7d5b5e0a13f8f84ec02442346cba0b842c6c81a6bf6176e64f3675c2ebf357cb5bb048e0b527bd39377c95681d22468da3d5de735 (@aaa.aaa.aaa.aaa) 1.05ms
[I 2021-06-12 17:18:10.067 JupyterHub log:189] 302 GET /static/components/react/react-dom.production.min.js?v=6fc58c1c4736868ff84f57bd8b85f2bdb985993a9392718f3b4af4bfa10fb4efba2b4ddd68644bd2a8daf0619a3844944c9c43f8528364a1aa6fc01ec1b8ae84 -> /hub/static/components/react/react-dom.production.min.js?v=6fc58c1c4736868ff84f57bd8b85f2bdb985993a9392718f3b4af4bfa10fb4efba2b4ddd68644bd2a8daf0619a3844944c9c43f8528364a1aa6fc01ec1b8ae84 (@aaa.aaa.aaa.aaa) 1.01ms
[W 2021-06-12 17:18:10.171 JupyterHub log:189] 404 GET /hub/static/components/react/react-dom.production.min.js?v=6fc58c1c4736868ff84f57bd8b85f2bdb985993a9392718f3b4af4bfa10fb4efba2b4ddd68644bd2a8daf0619a3844944c9c43f8528364a1aa6fc01ec1b8ae84 (@aaa.aaa.aaa.aaa) 1.02ms
[I 2021-06-12 17:18:10.278 JupyterHub log:189] 302 GET /static/components/create-react-class/index.js?v=894ad57246e682b4cfbe7cd5e408dcd6b38d06af4de4f3425991e2676fdc2ef1732cbd19903104198878ae77de12a1996de3e7da3a467fb226bdda8f4618faec -> /hub/static/components/create-react-class/index.js?v=894ad57246e682b4cfbe7cd5e408dcd6b38d06af4de4f3425991e2676fdc2ef1732cbd19903104198878ae77de12a1996de3e7da3a467fb226bdda8f4618faec (@aaa.aaa.aaa.aaa) 0.99ms
[W 2021-06-12 17:18:10.383 JupyterHub log:189] 404 GET /hub/static/components/create-react-class/index.js?v=894ad57246e682b4cfbe7cd5e408dcd6b38d06af4de4f3425991e2676fdc2ef1732cbd19903104198878ae77de12a1996de3e7da3a467fb226bdda8f4618faec (@aaa.aaa.aaa.aaa) 0.97ms
[I 2021-06-12 17:18:10.521 JupyterHub log:189] 302 GET /static/components/jquery/jquery.min.js?v=20210612171805 -> /hub/static/components/jquery/jquery.min.js?v=20210612171805 (@aaa.aaa.aaa.aaa) 0.95ms
[W 2021-06-12 17:18:10.625 JupyterHub log:189] 404 GET /hub/static/components/jquery/jquery.min.js?v=20210612171805 (@aaa.aaa.aaa.aaa) 1.41ms
^C[C 2021-06-12 17:18:15.594 JupyterHub app:2941] Received signal SIGINT, initiating shutdown...
[I 2021-06-12 17:18:15.595 JupyterHub app:2582] Cleaning up single-user servers...
[I 2021-06-12 17:18:15.595 JupyterHub proxy:772] Cleaning up proxy[4430]...
17:18:15.607 [ConfigProxy] warn: Terminated
[I 2021-06-12 17:18:15.809 JupyterHub batchspawner:431] Stopping server job 188
[I 2021-06-12 17:18:15.828 JupyterHub batchspawner:316] Cancelling job 188: sudo -E -u aerobinsoniv@contoso.com scancel 188
[I 2021-06-12 17:18:18.195 JupyterHub app:2614] ...done
```

[Jupyterhub.har.zip](https://github.com/jupyterhub/batchspawner/files/6673895/Jupyterhub.har.zip)