Azure / Azure/cyclecloud-gridengine

Manually added nodes in default config don't join cluster

Abierto
#25 0 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Python
Estrellas
7
Forks
18
Merge medio
5 d 21 h
PR fusionados (30 d)
2

Descripción

When manually adding a node, gridengine doesn't assume any hostgroup information.

2021-07-21 16:05:01,816 WARNING: No hostgroups were found for Node(htc-1, ip-0A010529, Standard_F2s_v2, 51ec3a08-c6ee-4af9-a25e-0855a54f2e36). Adding to @allhosts only. Please manually add this node to a hostgroup using qconf. If this was a manually started node, add 'gridengine.hostgroups = @preferred-hostgroup' to the nodearray's configuration section so that future nodes will automatically be added.
2021-07-21 16:05:01,867 ERROR: '/sched/sge/sge-2011.11/bin/linux-x64/qmod -e '*@ip-0A010529'' failed: Command '['/sched/sge/sge-2011.11/bin/linux-x64/qmod', '-e', '*@ip-0A010529']' returned non-zero exit status 1.
2021-07-21 16:05:01,868 WARNING: Could not add Node(htc-1, ip-0A010529, Standard_F2s_v2, 51ec3a08-c6ee-4af9-a25e-0855a54f2e36) to cluster: Command '['/sched/sge/sge-2011.11/bin/linux-x64/qmod', '-e', '*@ip-0A010529']' returned non-zero exit status 1.
2021-07-21 16:05:01,878 ERROR: '/sched/sge/sge-2011.11/bin/linux-x64/qmod -d '*@ip-0A010529'' failed: Command '['/sched/sge/sge-2011.11/bin/linux-x64/qmod', '-d', '*@ip-0A010529']' returned non-zero exit status 1.
2021-07-21 16:05:01,879 ERROR: b'invalid queue "*@ip-0A010529"\n'
Traceback (most recent call last):
File "/opt/cycle/gridengine/venv/lib/python3.6/site-packages/gridengine/driver.py", line 378, in _add_node_to_cluster
self.ge_env.qbin.qmod(["-e", "*@" + node.hostname])
File "/opt/cycle/gridengine/venv/lib/python3.6/site-packages/gridengine/qbin.py", line 142, in qmod
return self._call(_QMOD_PATH, args, check)
File "/opt/cycle/gridengine/venv/lib/python3.6/site-packages/gridengine/qbin.py", line 132, in _call
return check_output([bin] + args)
File "/opt/cycle/gridengine/venv/lib/python3.6/site-packages/gridengine/qbin.py", line 82, in check_output
return _check_output(cmd, *args, **kwargs).decode()
File "/usr/lib64/python3.6/subprocess.py", line 356, in check_output
**kwargs).stdout
File "/usr/lib64/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/sched/sge/sge-2011.11/bin/linux-x64/qmod', '-e', '*@ip-0A010529']' returned non-zero exit status 1.

Perhaps we should add a autoscale.json configuration so that manually added nodes (which don't have gridengine.hostgroups set can join hostgroups.

"default_hostgroups": [{"select": "node.nodearray": "hpc", "hostgroups": ["@cyclehpc"]},
{"select": "node.nodearray": "htc", "hostgroups": ["@cyclehtc"]}]

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Línea de trabajo

Comienza en _add_node_to_cluster en gridengine/driver.py e inspecciona la invocación de qmod; qbin.py muestra cómo se ejecuta ese comando. Compara el manejo de hostgroup para los nodos añadidos manualmente con la configuración de nodearray y los ajustes propuestos de default_hostgroups. Se considera terminado cuando un nodo añadido manualmente recibe el hostgroup previsto y se incorpora sin el error de cola no válida mostrado.

Escrito por el modelo de indexación a partir del texto del issue.

Evaluación

Stack tecnológico
python
Área
devops, distributed-systems
Tipo de issue
Error
Dificultad
4/5
Tiempo estimado
3-5 días
Estado de actividad
Estancado
Claridad
Bastante claro
Aptitud para principiantes
42/100

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.