Azure / Azure/azure-sdk-for-python

azure-core: prevent unsupported per-call kwargs from leaking to built-in transports

Open
#46,365 3 comments 0 reactions 4 assignees Claimed by @iscai-msft View on GitHub
Azure.Core Core.Http customer-reported feature-request needs-team-attention
Dominant language
Python
Stars
5.6k
Forks
3.4k
Avg merge
1d 21h
Merged PRs (30d)
193

Description

**Is your feature request related to a problem? Please describe.**

While addressing the `azure-data-tables` fix in #46137 for #46014, we identified a broader shared transport-layer issue in `azure-core`. Unsupported SDK-specific kwargs can leak through pipeline context into the built-in transports, where backend HTTP libraries may reject them with low-level `TypeError`s.

This surfaced in `azure-data-tables` async usage, where `list_entities(query_filter=...)` allowed unsupported kwargs to flow through `pipeline.run(..., **kwargs)` into `AioHttpTransport.send()` and ultimately `aiohttp.ClientSession.request()`.

Today, per-call kwargs are accumulated in `PipelineContext.options` and passed through to transport `send(..., **kwargs)`. The shared cleanup step removes only a small denylist of internal keys, so leftover SDK-specific kwargs can still reach the transport layer.

This creates a class of bugs where:
- async transports fail with backend-specific `TypeError`
- sync transports may appear to work or fail differently
- individual SDKs end up patching symptoms method by method

### Call path

- `sdk/tables/azure-data-tables/azure/data/tables/_generated/aio/operations/_operations.py:418`
`pipeline.run(request, stream=_stream, **kwargs)`
- `sdk/core/azure-core/azure/core/pipeline/_base_async.py:226`
`PipelineContext(self._transport, **kwargs)`
- `sdk/core/azure-core/azure/core/pipeline/__init__.py:66`
`PipelineContext.options = kwargs`
- `sdk/core/azure-core/azure/core/pipeline/_base_async.py:111`
`_AsyncTransportRunner.send()` forwards `**request.context.options` to transport `send(...)`
- `sdk/core/azure-core/azure/core/pipeline/_base.py:54`
`cleanup_kwargs_for_transport` only strips a few hardcoded keys
- `sdk/core/azure-core/azure/core/pipeline/transport/_aiohttp.py:352`
`session.request(..., **config)` forwards leftover kwargs to `aiohttp`
- `sdk/core/corehttp/corehttp/runtime/pipeline/_tools.py:50`
`corehttp` has an equivalent narrow helper

### Minimal reproducer

Concrete reproducer observed in `azure-data-tables`:

```python
import asyncio

from azure.data.tables.aio import TableServiceClient
from azure.identity.aio import DefaultAzureCredential

async def main() -> None:
endpoint = "https://.table.core.windows.net"
async with (
DefaultAzureCredential() as cred,
TableServiceClient(endpoint=endpoint, credential=cred) as svc,
):
table = svc.get_table_client("")
async for entity in table.list_entities(query_filter="PartitionKey eq 'mypartition'"):
print(entity)
break

asyncio.run(main())
```

Observed failure on the async path:

```text
TypeError: ClientSession._request() got an unexpected keyword argument 'query_filter'
```

The tables-specific PR fixes this one API by validating the unsupported kwarg earlier, but the broader leak path remains for any SDK-specific kwarg that survives into transport `**kwargs`.

**Describe the solution you'd like**

Investigate a shared fix in built-in transports that:
1. explicitly consumes the per-call kwargs each transport supports
2. detects leftover kwargs before forwarding to the underlying HTTP library
3. raises a clear SDK-level error for unsupported transport kwargs

Transports in scope:
- `AioHttpTransport`
- `RequestsTransport`
- `AsyncioRequestsTransport`
- `TrioRequestsTransport`

Related parity check:
- `corehttp` has a similar pipeline/transport sanitization pattern and may need the same treatment

**Describe alternatives you've considered**

- Expanding `cleanup_kwargs_for_transport` into a broad allowlist at the pipeline-runner level. Rejected because the runner serves both built-in and custom transports, and a static allowlist there risks silently dropping legitimate kwargs for custom transport implementations.
- Fixing the leak in generator templates so operations don't forward `**kwargs` to `pipeline.run()`. Reasonable as a longer-term improvement, but it does not address existing hand-written SDKs or the shared transport contract.
- Per-SDK point fixes, as in #46137. Correct for specific API contracts, but they do not prevent the broader bug class.

**Additional context**

Sync compatibility needs care. Some stray kwargs are currently tolerated on sync paths in ways that async paths are not, so a systemic fix may need a warn-then-error transition rather than an immediate hard break.

This issue is not proposing to change service-level API contracts. Individual SDKs should still validate unsupported public kwargs at their own API boundaries where appropriate, as done in #46137 for `TableClient.list_entities`.

## References

- Source PR: #46137
- Original report: #46014
- Systemic concern raised by: @rgant

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.