apache / apache/airflow

Add configuration support for aiohttp.ClientSession kwargs (e.g. trust_env) in deferrable operators / triggerer

Open
#67,893 4 comments 0 reactions 0 assignees View on GitHub
area:async-operators area:core kind:feature needs-triage
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 9h
Merged PRs (30d)
472

Description

### Description

Airflow should provide a global supported config mechanism to pass additional kwargs to aiohttp.ClientSession at construction time. For example, an Airflow config key — e.g. `[aiohttp] trust_env = True` in airflow.cfg, read by a shared session factory used across all providers.

### Use case/motivation

## Problem

Deferrable operators that use `aiohttp` internally (e.g. `SnowflakeSqlApiOperator`, and others backed by async HTTP clients) create `aiohttp.ClientSession` instances without passing `trust_env=True`. This means they silently ignore proxy-related environment variables (`HTTPS_PROXY`, `HTTP_PROXY`, `NO_PROXY`) set in the execution environment.

In managed environments like AWS MWAA — where corporate proxy settings are injected via a startup script into environment variables — this causes triggers to bypass the proxy entirely and time out, while synchronous operators using requests or urllib work correctly.

## Current workaround

You can set `trust_env` globally via a Monkey-patch of `aiohttp.ClientSession.__init__` in `airflow_local_settings.py`:

```python
import aiohttp
from typing import Any

_original_init = aiohttp.ClientSession.__init__

def _trust_env_init(self: aiohttp.ClientSession, *args: Any, **kwargs: Any) -> None:
kwargs.setdefault("trust_env", True)
_original_init(self, *args, **kwargs)

aiohttp.ClientSession.__init__ = _trust_env_init
```
This is fragile, not discoverable, and should not be necessary for what is effectively a standard proxy configuration requirement.

## Environment

Version: Apache Airflow 3.x (probably also affects 2.x)
Provider: apache-airflow-providers-snowflake (and any other provider using aiohttp in triggers)
Deployment: AWS MWAA (corporate proxy injected via startup script env vars)

## Impact

Any organisation running Airflow behind a corporate proxy will hit this silently — deferrable operators time out with no clear indication that proxy settings are being ignored.

### Related issues

_No response_

### Are you willing to submit a PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.