ipython / ipython/ipyparallel

PBSControllerLauncher: Unable to connect_client_sync()

Abierto
#619 16 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Jupyter Notebook
Estrellas
2.6k
Forks
1k
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

I have been using ipyparallel 6 for a while and would like to migrate to ipyparallel 7 mainly due to fact that the new Cluster API enables you to manage the entire process through a Jupyter Notebook. Unfortunatelly, I am having difficulties to connect a client to my cluster.

I have created a new IPython profile adding a custom `ipcluster_config.py`, which is a modified version of my existing/working config for ipp 6 (see below).

I can successfully start a cluster spawning two PBS jobs (controller and engine).

```python
import ipyparallel as ipp

cluster=ipp.Cluster(
n=128,
controller_ip='*',
profile='pbs'
)
```
```
# Using existing profile dir: '/network/datamic/home/lukask/.aixvipmap/.ipython/profile_pbs'
```
```python
await cluster.start_cluster()
```
```
# Job submitted with job id: '23777'
# Starting 4 engines with
# Job submitted with job id: '23778'
# , engine_sets=['1635407070'])>
```

But if I run the following line, the notebook will only show that the kernel is busy and it will never actually finish.

```python
rc = cluster.connect_client_sync()
```

Am I using the API incorrectly? What might be the problem?

### ipcluster_config.py

```python
c.Cluster.engine_launcher_class = 'ipyparallel.cluster.launcher.PBSEngineSetLauncher'

c.Cluster.controller_launcher_class = 'ipyparallel.cluster.launcher.PBSControllerLauncher'

c.PBSControllerLauncher.batch_template = '''
#PBS -N ipcontroller
#PBS -j oe
#PBS -l walltime=01:00:00
#PBS -l nodes=1:ppn=1

cd $PBS_O_WORKDIR

conda activate ipp7

ipcontroller --profile-dir={profile_dir}
'''

c.PBSEngineSetLauncher.batch_template = '''
#PBS -N ipengine
#PBS -j oe
#PBS -l walltime=01:00:00
#PBS -l nodes={n//4}:ppn=4

cd $PBS_O_WORKDIR

conda activate ipp7

module load intel
mpiexec -n {n} ipengine --profile-dir={profile_dir}
'''
```

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.