equinor / equinor/ert

Handling of `QUEUE_OPTION [..] QUEUE` in Scheduler

Open
#7,112 7 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
161
Forks
140
Avg merge
2d 1h
Merged PRs (30d)
138

Description

We can make the following assumptions:
1. Each HPC system has a way to choose a named queue.
2. Each HPC system has a default queue it chooses. Ie, `qsub /usr/bin/true` will execute on _some_ queue even though it's not specified.
3. The user may enter their preferred queue, which the driver must attempt to use.
4. The user may enter incorrect information.

The `LocalDriver` is an exception, but we may pretend it has a queue called `local`.

We may check whether the queue is valid and exit early if there is an issue with their chosen queue. To do this, we should extend `Driver` with the following method:
```py
async def use_queue(self, queue_name: str) -> None:
"""
Submit jobs to `queue_name` queue.

Raises:
ValueError if there is an issue with the queue
"""
```

For `LocalDriver` this function does nothing. `LSFDriver` may run `bqueues` to verify that the user's queue exists, and raise `ValueError` (or an appropriate exception) if it doesn't.

This makes it possible to check that the queue seems okay long before we submit any jobs. Maybe we can have a `@classmethod` function `check_queue` which is ran when the GUI starts up, so we can show an error message to the user.

Contributor guide

Open the contributing guide

Research direction

Start by locating the Driver, LocalDriver, and LSFDriver implementations and the scheduler handling for QUEUE_OPTION. Read how the GUI starts and how LSF can invoke bqueues, then determine the intended queue-selection and validation flow. Done means the queue behavior and error handling are agreed for local and HPC drivers before implementation begins.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, hpc
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.