Azure / Azure/azure-sdk-for-python
azure-core: prevent unsupported per-call kwargs from leaking to built-in transports
- Dominant language
- Python
- Stars
- 5.6k
- Forks
- 3.4k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 193
Description
**Is your feature request related to a problem? Please describe.**
While addressing the `azure-data-tables` fix in #46137 for #46014, we identified a broader shared transport-layer issue in `azure-core`. Unsupported SDK-specific kwargs can leak through pipeline context into the built-in transports, where backend HTTP libraries may reject them with low-level `TypeError`s.
This surfaced in `azure-data-tables` async usage, where `list_entities(query_filter=...)` allowed unsupported kwargs to flow through `pipeline.run(..., **kwargs)` into `AioHttpTransport.send()` and ultimately `aiohttp.ClientSession.request()`.
Today, per-call kwargs are accumulated in `PipelineContext.options` and passed through to transport `send(..., **kwargs)`. The shared cleanup step removes only a small denylist of internal keys, so leftover SDK-specific kwargs can still reach the transport layer.
This creates a class of bugs where:
- async transports fail with backend-specific `TypeError`
- sync transports may appear to work or fail differently
- individual SDKs end up patching symptoms method by method
### Call path
- `sdk/tables/azure-data-tables/azure/data/tables/_generated/aio/operations/_operations.py:418`
`pipeline.run(request, stream=_stream, **kwargs)`
- `sdk/core/azure-core/azure/core/pipeline/_base_async.py:226`
`PipelineContext(self._transport, **kwargs)`
- `sdk/core/azure-core/azure/core/pipeline/__init__.py:66`
`PipelineContext.options = kwargs`
- `sdk/core/azure-core/azure/core/pipeline/_base_async.py:111`
`_AsyncTransportRunner.send()` forwards `**request.context.options` to transport `send(...)`
- `sdk/core/azure-core/azure/core/pipeline/_base.py:54`
`cleanup_kwargs_for_transport` only strips a few hardcoded keys
- `sdk/core/azure-core/azure/core/pipeline/transport/_aiohttp.py:352`
`session.request(..., **config)` forwards leftover kwargs to `aiohttp`
- `sdk/core/corehttp/corehttp/runtime/pipeline/_tools.py:50`
`corehttp` has an equivalent narrow helper
### Minimal reproducer
Concrete reproducer observed in `azure-data-tables`:
```python
import asyncio
from azure.data.tables.aio import TableServiceClient
from azure.identity.aio import DefaultAzureCredential
async def main() -> None:
endpoint = "https://.table.core.windows.net"
async with (
DefaultAzureCredential() as cred,
TableServiceClient(endpoint=endpoint, credential=cred) as svc,
):
table = svc.get_table_client("")
async for entity in table.list_entities(query_filter="PartitionKey eq 'mypartition'"):
print(entity)
break
asyncio.run(main())
```
Observed failure on the async path:
```text
TypeError: ClientSession._request() got an unexpected keyword argument 'query_filter'
```
The tables-specific PR fixes this one API by validating the unsupported kwarg earlier, but the broader leak path remains for any SDK-specific kwarg that survives into transport `**kwargs`.
**Describe the solution you'd like**
Investigate a shared fix in built-in transports that:
1. explicitly consumes the per-call kwargs each transport supports
2. detects leftover kwargs before forwarding to the underlying HTTP library
3. raises a clear SDK-level error for unsupported transport kwargs
Transports in scope:
- `AioHttpTransport`
- `RequestsTransport`
- `AsyncioRequestsTransport`
- `TrioRequestsTransport`
Related parity check:
- `corehttp` has a similar pipeline/transport sanitization pattern and may need the same treatment
**Describe alternatives you've considered**
- Expanding `cleanup_kwargs_for_transport` into a broad allowlist at the pipeline-runner level. Rejected because the runner serves both built-in and custom transports, and a static allowlist there risks silently dropping legitimate kwargs for custom transport implementations.
- Fixing the leak in generator templates so operations don't forward `**kwargs` to `pipeline.run()`. Reasonable as a longer-term improvement, but it does not address existing hand-written SDKs or the shared transport contract.
- Per-SDK point fixes, as in #46137. Correct for specific API contracts, but they do not prevent the broader bug class.
**Additional context**
Sync compatibility needs care. Some stray kwargs are currently tolerated on sync paths in ways that async paths are not, so a systemic fix may need a warn-then-error transition rather than an immediate hard break.
This issue is not proposing to change service-level API contracts. Individual SDKs should still validate unsupported public kwargs at their own API boundaries where appropriate, as done in #46137 for `TableClient.list_entities`.
## References
- Source PR: #46137
- Original report: #46014
- Systemic concern raised by: @rgant
Contributor guide
Assessment
This issue has not been assessed yet.