apache / apache/airflow

Remove executor config pickling -- and make note of other pickling usages as well

Open
#47,059 1 comment 0 reactions 0 assignees View on GitHub
area:Executors-core kind:feature kind:meta
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 10h
Merged PRs (30d)
483

Description

### Body

Shall we remove pickling of executor config?

It's a bit of a breaking change but, who uses that anyway?

While you're at it, please take a look and see if there's anything else that we are pickling that we could ... stop pickling (such as xcom etc) and please create issues for that as well / bring to dev list.

### Committer

- [x] I acknowledge that I am a maintainer/committer of the Apache Airflow project.

Contributor guide

Open the contributing guide

Research direction

Start by locating how executor configuration is pickled and tracing other pickling usages, including XCom. Determine the compatibility impact of removing executor-config pickling, then document other candidates and create follow-up issues or bring them to the dev list. Done means the executor-config decision is implemented and the remaining usages are recorded for separate discussion.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.