apache / apache/airflow

Add support for preinjecting connection for bundles

Open
#46,429 3 comments 1 reaction 0 assignees View on GitHub
area:API area:dag-bundles kind:feature kind:meta
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 10h
Merged PRs (30d)
483

Description

### Body

Some bundles will end up needing a connection - we should preinject it so we don't have to hit the execution api right away when starting a task. We will have to do it in a generic way though, since not all bundles will rely on connections.

We can send it as part of [this response](https://github.com/apache/airflow/blob/3c732fbc088b9baa849b7a53c307642a9f4328ec/airflow/api_fastapi/execution_api/datamodels/taskinstance.py#L230-L243) to [this request](https://github.com/apache/airflow/blob/3c732fbc088b9baa849b7a53c307642a9f4328ec/task_sdk/src/airflow/sdk/execution_time/supervisor.py#L611-L625).

This can happen post 3.0.

### Committer

- [x] I acknowledge that I am a maintainer/committer of the Apache Airflow project.

Contributor guide

Open the contributing guide

Research direction

Start with the response model in airflow/api_fastapi/execution_api/datamodels/taskinstance.py around lines 230-243 and the request handling in task_sdk/src/airflow/sdk/execution_time/supervisor.py around lines 611-625. Trace how task startup obtains connections and determine how an optional, generic bundle payload can be passed through. Done means connection-dependent bundles can start without an immediate execution API call while other bundles continue to work.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.