ExecutionGraph is un-pickleable
Open
@FrankD412 is already working on this.
Since Jun 24, 2020.
bug
Confirmed
- Dominant language
- Python
- Stars
- 161
- Forks
- 49
- PR merge metrics
- No merged PRs in 30d
Description
In Maestro 1.1.7 and 1.1.8, pickling DAG objects that have the dynamically added function `_pass_detect_cycle` breaks pickle, which is causing distributed merlin jobs to break, since merlin's celery config is to use pickle to serialize and deserialize tasks.
Possible solution: Remove _pass_detect_cycle, and instead add an optional bool parameter to `ExecutionGraph` that defaults to `True`, but when `False` skips cycle checks when new nodes are added.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.