apache / apache/airflow

Enable Presets to pre-load parameters for the same DAG

Open
#71,285 1 comment 0 reactions 0 assignees View on GitHub
area:UI kind:feature needs-triage
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 9h
Merged PRs (30d)
472

Description

### Description

In a DAG we can currently specify different params and their default value however I'd like to have a list of presets from which I can load on the fly different pre-configured default values and load them on the fly and eventually edit the preset's default values from the UI if needed.

### Use case/motivation

I often need to to run Machine learning training experiments with the same model architecture but different dataset for different tasks, since what changes is the dataset but not the architectures I use a dag to fetch data, train, evaluated etc. However different datasets need different augmentations techniques which often have lots of parameters and having multiple presets could help engineers start the training with a preselected config and maybe tweak only few parameters rather than rewriting the whole augmentations

### Related issues

_No response_

### Are you willing to submit a PR?

- [x] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start by reviewing how DAG parameters and their defaults are currently represented and exposed in the UI. Clarify the preset lifecycle, including loading values on demand and editing preset defaults, then define completion around selecting a preset and changing its values for a DAG run.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.