cdklabs / cdklabs/cdk-pipelines-github
Pipelines with large job graphs fail at GitHub workflow startup (4700+ dependency edges)
- Dominant language
- TypeScript
- Stars
- 384
- Forks
- 45
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 4
Description
## Description
When `cdk-pipelines-github` generates a workflow that deploys across a large number of AWS accounts (in our case ~50), the resulting YAML (~9,000 lines, ~160 jobs, 4,700+ `needs:` dependency edges) fails immediately at workflow startup — before any job executes.
GitHub Support confirmed this is caused by an undocumented internal database storage limit on job dependency graphs. The constraint is based on the combined size of all job names and their `needs` references, not just job count alone.
## Error behaviour
- Workflow run is created but immediately transitions to a failed state
- No jobs execute
- No error message is surfaced in the GitHub Actions UI — the failure happens before any runner is allocated
## Root cause (per GitHub Support)
> "Your workflow contains 160 jobs with approximately 4,700+ dependency edges across them, generating a job dependency graph that exceeds an internal database storage limit. This causes the run to fail at startup before any job executes."
This is a known issue tracked internally by GitHub Engineering. Their intended long-term fix is to gracefully handle oversized graphs (skip UI graph rendering rather than failing the run), but it has not shipped yet.
## Context
- Pipeline deploys to ~50 AWS accounts in a single generated workflow file
- The workflow is entirely auto-generated by `cdk-pipelines-github` — the YAML cannot be edited directly
- There is no documented GitHub limit for job count or dependency edges
## Workaround suggested by GitHub Support
Split the generated workflow into multiple smaller workflow files and chain them using the `workflow_run` trigger to preserve stage ordering. For example:
```yaml
on:
workflow_run:
workflows: ["InfraStage"]
types: [completed]
```
Since the YAML is auto-generated, this change must be implemented in the CDK pipeline source.
## Proposed solution
Add support for splitting a pipeline across multiple workflow files, with stages or wave boundaries acting as split points. Possible approaches:
- A `splitWorkflowAt` option (stage name or wave index) that emits separate `.github/workflows/` files and wires them together via `workflow_run` triggers automatically
- An automatic split when job count or estimated edge count exceeds a configurable threshold
- At minimum, documentation of this limit and a manual escape hatch for users to define workflow boundaries
## Environment
- ~50 AWS accounts / ~160 generated jobs / ~4,700 `needs` edges
- Generated YAML: ~9,000 lines
- GitHub confirmed the issue via Support ticket
Contributor guide
Assessment
This issue has not been assessed yet.