I want to write my pipeline in Python so I can create stages programmatically
- Dominant language
- Python
- Stars
- 60
- Forks
- 18
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 38
Description
For example:
```python
from calkit.models import Pipeline, PythonScriptStage, SBatchStage
def make_nsys_stage(name: str) -> SBatchStage:
return SBatchStage(name=name, ...)
pipeline = Pipeline(
stages=[
make_nsys_stage("baseline"),
make_nsys_stage("mod"),
make_nsys_stage("const"),
PythonScriptStage(
name="summarize",
script="scripts/summarize.py",
inputs=["results/nsys/baseline.sqlite"],
),
]
)
```
Then in `calkit.yaml`:
```yaml
pipeline:
kind: python
path: pipeline.py
```
Calkit will import the script and look for a single Pipeline object in `locals`, which it will then use to compile the pipeline.
Related to #561
We could also simply use a `calkit.py` file and merge it with `calkit.yaml`, overriding any names.
Contributor guide
Assessment
This issue has not been assessed yet.