Remove executor config pickling -- and make note of other pickling usages as well
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 483
Description
### Body
Shall we remove pickling of executor config?
It's a bit of a breaking change but, who uses that anyway?
While you're at it, please take a look and see if there's anything else that we are pickling that we could ... stop pickling (such as xcom etc) and please create issues for that as well / bring to dev list.
### Committer
- [x] I acknowledge that I am a maintainer/committer of the Apache Airflow project.
Contributor guide
Research direction
Start by locating how executor configuration is pickled and tracing other pickling usages, including XCom. Determine the compatibility impact of removing executor-config pickling, then document other candidates and create follow-up issues or bring them to the dev list. Done means the executor-config decision is implemented and the remaining usages are recorded for separate discussion.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100