MetOffice / MetOffice/CSET

Go back to running bake as individual tasks rather than using rose bunch

Open
#1,707 11 comments 0 reactions 2 assignees Claimed by @james-a-f-ross View on GitHub
enhancement R2O Trials
Dominant language
Python
Stars
33
Forks
19
Avg merge
1d 22h
Merged PRs (30d)
30

Description

### What problem does your feature request solve?
Running a full length trials for a single UKV parameter causes cset bake to run out of wall clock, with a message that the out of memory (OOM) killer has been run multiple times.

There is also the issue of user control. When tasks are grouped in to a bunch it is difficult to see which specific task has caused an issue.

### Describe the solution you'd like
I'd like to see the workflow to run the plotting in individual cylc tasks. This will allow more memory/wall clock for individual tasks and will allow users to intervene if a specific task fails.

### Describe alternatives you've considered
Changing the number of tasks that `rose bunch` runs at once. This would ultimately turn a bunch of parallel tasks in to sequential tasks and significantly slow down cset.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.