[Proposal] YAML Specification Improvements
- Dominant language
- Python
- Stars
- 161
- Forks
- 49
- PR merge metrics
- No merged PRs in 30d
Description
So I've been looking at some of the limitations of the YAML specification as it is, and I've come up with a new structure for it.
```
name: step_name
description: Some human readable English here for the step.
cmd: |
exec some commands here
restart: |
exec a restart here
post: |
exec a small-ish local step after the main step terminates
resources:
sched:
nodes:
tasks:
procs:
gpus:
cores per task:
```
The idea behind splitting out the resources is two fold:
1. This organization makes it so that Maestro can have monikers like `$(resource.)` and so long as the key exists in `resources:` it can be used. Users could then "manually" select which MPI they want to use in a fashion similar to `mpirun -n $(resources.tasks) ... some --command --to run`. Principally, user could even do shell math to computer MPI related values (and Maestro could even calculate high level settings inferring from statically set values).
2. Consolidation of resource settings and the ability to allow for more flexible step-wise specification of required resources. Sensible defaults could be provided, and a higher level batch could be specified, and then override values with step-wise values. It opens up a clear and simple line of application of batch parameters that can be consolidated in a uniform fashion while still giving the user more control.
Contributor guide
Assessment
This issue has not been assessed yet.