Add structural `Sequence` and `Independent` composite models
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 44
- Forks
- 7
- Avg merge
- 17h 16m
- Merged PRs (30d)
- 13
Description
Add two composite CallableModels for expressing the structural shape of a workflow.
Sequenceholds an ordered list of childCallableModels and declares that they must execute one after another. Useful when ordering matters because of external side effects — for example, an ETL stage that must finish writing before the next stage reads, or a migration that must complete before a backfill.Independentholds a list of childCallableModels and declares that they have no ordering relationship to one another. Whether they actually execute concurrently, in parallel across processes, distributed across workers, or simply one after another is left to the evaluator.
Data dependencies between steps are expressed the existing ccflow way: by composing models so that one contains another. These two primitives only declare ordering constraints, and can be freely nested (a Sequence of Independents of Sequences, etc.) to build arbitrary DAGs. The resulting graph is itself a CallableModel.
Execution policy — concurrency, scheduling, distribution, retries — is the evaluator's responsibility. The same Sequence/Independent graph should run unchanged under the existing GraphEvaluator, a future Ray-based evaluator, a future Celery-based evaluator, etc.
Open questions
- Naming.
Independentcaptures the structural property accurately but is less immediately recognizable than something likeParallel. Worth a final naming pass before settling. - Result shape. Should the composite return a list keyed by position, a dict keyed by child name, or both via a configurable option? Names are friendlier for downstream consumers but require unique identifiers per child.
- Failure semantics. For
Independent, should a single child failure prevent siblings from being scheduled (fail-fast) or let them complete and report all errors? ForSequence, fail-fast is the obvious default but worth confirming. In either case the actual cancellation/scheduling behavior is the evaluator's call; the composite only declares the policy.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No files or tests are named. Start by reading the existing CallableModel and GraphEvaluator implementations to understand composition and evaluation boundaries. Before implementation, resolve the open questions around naming, result shape, and Independent failure semantics; done means nested Sequence and Independent models express ordering without embedding evaluator-specific scheduling.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100