Point72 / Point72/ccflow

Add structural `Sequence` and `Independent` composite models

Open
#229 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
44
Forks
7
Avg merge
17h 16m
Merged PRs (30d)
13

Description

Add two composite CallableModels for expressing the structural shape of a workflow.

  • Sequence holds an ordered list of child CallableModels and declares that they must execute one after another. Useful when ordering matters because of external side effects — for example, an ETL stage that must finish writing before the next stage reads, or a migration that must complete before a backfill.
  • Independent holds a list of child CallableModels and declares that they have no ordering relationship to one another. Whether they actually execute concurrently, in parallel across processes, distributed across workers, or simply one after another is left to the evaluator.

Data dependencies between steps are expressed the existing ccflow way: by composing models so that one contains another. These two primitives only declare ordering constraints, and can be freely nested (a Sequence of Independents of Sequences, etc.) to build arbitrary DAGs. The resulting graph is itself a CallableModel.

Execution policy — concurrency, scheduling, distribution, retries — is the evaluator's responsibility. The same Sequence/Independent graph should run unchanged under the existing GraphEvaluator, a future Ray-based evaluator, a future Celery-based evaluator, etc.

Open questions
  • Naming. Independent captures the structural property accurately but is less immediately recognizable than something like Parallel. Worth a final naming pass before settling.
  • Result shape. Should the composite return a list keyed by position, a dict keyed by child name, or both via a configurable option? Names are friendlier for downstream consumers but require unique identifiers per child.
  • Failure semantics. For Independent, should a single child failure prevent siblings from being scheduled (fail-fast) or let them complete and report all errors? For Sequence, fail-fast is the obvious default but worth confirming. In either case the actual cancellation/scheduling behavior is the evaluator's call; the composite only declares the policy.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files or tests are named. Start by reading the existing CallableModel and GraphEvaluator implementations to understand composition and evaluation boundaries. Before implementation, resolve the open questions around naming, result shape, and Independent failure semantics; done means nested Sequence and Independent models express ordering without embedding evaluator-specific scheduling.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.