DDMAL / DDMAL/hpc-trainer-component

Calvo HPC job not getting into Cedar's queue

Open
#11 1 comment 0 reactions 0 assignees View on GitHub
help wanted high priority
Dominant language
Python
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

Calvo HPC job (real name: `Training Model for Patchwise Analysis of Music Document - HPC`) got stuck in `processing`. Usually, this job takes around 4–5 hours to complete, now it got stuck in processing for more than 12 hours and it won't finish.

When I went to check the queue of jobs in Cedar, this job was nowhere to be found. For some reason, running the job in rodan2 is not getting the job into the Cedar queue and, therefore, it is never executed by Cedar. @JRegimbal suspects that this could be an issue with the `check.py` script. We have to make sure that `check.py` is being run frequently from cedar (since this script checks if there is anything in Rodan that needs to be handled). Only the person in charge of that account can check this (we suspect that might be @deepio?)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.