Sub-DAG output variable propagation for parallel execution
- Dominant language
- Go
- Stars
- 4k
- Forks
- 332
- Avg merge
- 19h 26m
- Merged PRs (30d)
- 133
Description
## Problem
When a child sub-DAG sets `output:` on its steps, the parent's parallel results JSON shows `"outputs": {}` for every child. There is no way for a parent DAG to read collected outputs from parallel sub-DAG invocations.
## Expected Behavior
Child sub-DAG `output:` variables should be propagated to the parent and accessible from the parallel step's results. The parent should be able to reference aggregated outputs from all parallel children.
## Example
```yaml
# Parent
- id: fan_out
call: child-dag
parallel: ${ITEMS}
# After completion, ${fan_out.outputs} should be a JSON array
# of each child's output variables
- id: collect
depends: [fan_out]
command: echo "${fan_out.outputs}"
```
## Motivation
Fan-out/fan-in is a core DAG pattern. Without output propagation, the only way to pass data from parallel children back to the parent is through side-effect files (temp files on disk), which is fragile and hard to coordinate.
## Testing
- PR must include integration test under intg package
Contributor guide
Research direction
Start by tracing the parallel sub-DAG execution and result aggregation described in the issue, then inspect the intg package for existing integration-test patterns. Add coverage for a child step with output variables invoked through a parent parallel step. Done means the parent’s parallel results expose an aggregated JSON array that the following step can reference.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend, devops
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100