MetOffice / MetOffice/CMEW

Investigate the use of external triggers to await completion of the CDDS workflow

Open
#158 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

investigation standardise
Dominant language
Python
Stars
5
Forks
3
Avg merge
5d 6h
Merged PRs (30d)
8

Description

In issue #111 we use the `--no-detach` option to launch the cylc scheduler for the CDDS convert workflow within the `standarise_model_data` task. This ensures the task is not successful until the CDDS convert workflow finishes.

An alternative approach to consider, that may be more robust, would be to use [cylc external triggers](https://cylc.github.io/cylc-doc/stable/html/user-guide/writing-workflows/external-triggers.html). There is some discussion of the potential approaches in the comments of #111.

Time box:
- 0.5 day

Outcome:
- Using cylc external triggers to check if the CDDS workflow has failed/succeeded/stalled
- Investigate CDDS time out, logging and status visibility as listed on #354

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading issue #111 and its comments, then review the Cylc external triggers documentation. Inspect the standardise_model_data task and its current --no-detach usage. Done means assessing external triggers for CDDS success, failure, and stalled states, including timeout, logging, and status visibility concerns from #354.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devops
Issue type
Feature
Difficulty
4/5
Estimated time
Half a day
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.