jupyterhub / jupyterhub/batchspawner

Error: Unable to parse job ID from text

Open
#148 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
207
Forks
136
PR merge metrics
No merged PRs in 30d

Description

Hi, I've got a slurm cluster with Kerberos and I'm running jupyterhub as non-root.
My jupyterhub_config file looks like this:
```python3
c.JupyterHub.authenticator_class = 'jupyterhub.auth.PAMAuthenticator'
c.PAMAuthenticator.open_sessions = False

#------------------------------------------------------------------------------
# Spawner config
#------------------------------------------------------------------------------
# We're using SLURM:
# - use 'batchspawner.SlurmSpawner' as spawner class
# - use 'wrapspawner.ProfileSpawner' to provide multiple spawner profiles which
# overwrite the default values
#

c.JupyterHub.spawner_class = 'batchspawner.SlurmSpawner'
#c.Spawner.http_timeout = 120
c.BatchSpawnerBase.req_nprocs = '2'
#c.BatchSpawnerBase.req_queue = 'jupyter'
#c.BatchSpawnerBase.req_host = 'mesabi.xyz.edu'
c.BatchSpawnerBase.req_runtime = '12:00:00'
c.BatchSpawnerBase.req_memory = '12gb'

# in slurm queues are called 'partitions'
c.SlurmSpawner.req_partition = 'jupyter'

c.SlurmSpawner.start_timeout = 240

c.SlurmSpawner.batch_script = '''#!/bin/bash
#SBATCH --partition={partition}
#SBATCH --time={runtime}
#SBATCH --output={homedir}/logs/jupyterhub_slurmspawner_%j.log
#SBATCH --job-name=jupyterhub-spawner
#SBATCH --cpus-per-task={nprocs}
#SBATCH --workdir={homedir}
#SBATCH --mem={memory}
#SBATCH {options}

# load necessary modules
module load anaconda/3-2018.12

# print environment
env

{cmd}
'''

c.JupyterHub.spawner_class = 'wrapspawner.ProfilesSpawner'
#c.Spawner.http_timeout = 120

c.ProfilesSpawner.profiles = [
(
'SLURM CPU only node - 2 core, 12 GB, 48 hours',
'slurm-2cpu',
'batchspawner.SlurmSpawner',
dict(
req_nprocs='2',
req_partition='slurm-cpu',
req_runtime='48:00:00',
req_memory='12000'
)
),
...
(
"Test server",
'local-test',
'jupyterhub.spawner.LocalProcessSpawner',
{
'ip':'0.0.0.0'
}
)
]

```

However, I cannot spawn sessions due to an error with parsing `auks` output:
```
[I 2019-07-07 03:08:22.789 JupyterHub batchspawner:188] Spawner submitting job using sudo -E -u sbatch --parsable
[I 2019-07-07 03:08:22.789 JupyterHub batchspawner:189] Spawner submitted script:
#!/bin/bash
#SBATCH --partition=slurm-cpu
#SBATCH --time=48:00:00
#SBATCH --output=/home//logs/jupyterhub_slurmspawner_%j.log
#SBATCH --job-name=jupyterhub-spawner
#SBATCH --cpus-per-task=2
#SBATCH --workdir=/home/
#SBATCH --mem=12000
#SBATCH
####SBATCH --uid=
# load necessary modules
module load anaconda/3-2018.12
# print environment
env
jupyterhub-singleuser --port=45147 --NotebookApp.default_url=/lab
Sun Jul 7 03:08:22 2019 [INFO4] [euid=5534,pid=5818] auks_krb5_cred: kerberos authentication context successfully initialized
Sun Jul 7 03:08:23 2019 [INFO4] [euid=5534,pid=5818] auks_api: auks cred added using default file
132826
[E 2019-07-07 03:08:23.061 JupyterHub batchspawner:509] SlurmSpawner unable to parse job ID from text: Sun Jul 7 03:08:22 2019 [INFO4] [euid=5534,pid=5818] auks_krb5_cred: kerberos context successfully initialized
Sun Jul 7 03:08:23 2019 [INFO4] [euid=5534,pid=5818] auks_api: auks cred added using default file
132826
[E 2019-07-07 03:08:23.061 JupyterHub batchspawner:195] Job submission failed with exit code Sun Jul 7 03:08:22 2019 [INFO4] [euid=5534,pid=5818] auks_krb5_cred: kerberos context successfully initialized
Sun Jul 7 03:08:23 2019 [INFO4] [euid=5534,pid=5818] auks_api: auks cred added using default file
132826
[E 2019-07-07 03:08:23.062 JupyterHub user:626] Unhandled error starting 's server:
[E 2019-07-07 03:08:23.081 JupyterHub pages:209] Failed to spawn single-user server with form
Traceback (most recent call last):
File "/opt/modules/anaconda/3-2018.12/lib/python3.7/site-packages/jupyterhub/handlers/pages.py", line 206, in post
await self.spawn_single_user(user, server_name=server_name, options=options)
File "/opt/modules/anaconda/3-2018.12/lib/python3.7/site-packages/jupyterhub/handlers/base.py", line 889, in spawn_single_user
timedelta(seconds=self.slow_spawn_timeout), finish_spawn_future
File "/opt/modules/anaconda/3-2018.12/lib/python3.7/site-packages/jupyterhub/handlers/base.py", line 807, in finish_user_spawn
await spawn_future
File "/opt/modules/anaconda/3-2018.12/lib/python3.7/site-packages/jupyterhub/user.py", line 642, in spawn
raise e
File "/opt/modules/anaconda/3-2018.12/lib/python3.7/site-packages/jupyterhub/user.py", line 546, in spawn
url = await gen.with_timeout(timedelta(seconds=spawner.start_timeout), f)
File "/opt/modules/anaconda/3-2018.12/lib/python3.7/site-packages/batchspawner/batchspawner.py", line 309, in start
assert len(self.job_id) > 0
AssertionError
```

Nevertheless, the test-server config works flawlessly.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.