aio-libs / aio-libs/aiohttp

Tracing has a weird integration with Client Middlewares

Open
#11,438 1 comment 1 reaction 1 assignee Claimed by @bdraco View on GitHub
Dominant language
Python
Stars
16.5k
Forks
2.4k
Avg merge
17h 22m
Merged PRs (30d)
212

Description

### Discussed in https://github.com/aio-libs/aiohttp/discussions/11437

Originally posted by **UltimateLobster** August 19, 2025
After testing abit with the new client middlewares feature I've noticed something weird about the integration of the feature with tracing.

It seems that the tracing hooks for `on_request_start`, `on_request_end` and `on_request_exception` are called before the client middlewares. It seems weird to me as I would expect tracing hooks to be as close to the actual sent requests as possible.

That means that if I have a middleware that modifies the requests, or retries the requests multiple times, it will not be reflected in the traces. Here I've created a minimal retry middleware and trace configs to reflect the issue.

```python
import asyncio

from types import SimpleNamespace
from aiohttp import (
ClientSession,
TraceConfig,
TraceRequestStartParams,
TraceRequestEndparams,
TraceExceptionParams,
ClientRequest,
ClientHandlerType,
ClientResponse
)

async def on_request_start(session: ClientSession, context: SimpleNamespace, params: TraceRequestStartParams) -> None:
print("on request start")

async def on_request_end(session: ClientSession, context: SimpleNamespace, params: TraceRequestEndParams) -> None:
print("on request end")

async def on_request_exception(session: ClientSession, context: SimpleNamespace, params: TraceRequestExceptionParams) -> None:
print("on request exception")

trace_config = TraceConfig()
trace_config.on_request_start.append(on_request_start)
trace_config.on_request_end.append(on_request_end)
trace_config.on_request_exception.append(on_request_exception)

async def middleware(request: ClientRequest, handler: ClientHandlerType) -> ClientResponse:
print("middleware start")
for i in range(3):
print(f"attempt {i}")
response = await handler(request)

if response.ok:
break

print("middleware end")
return response

async def main():
async with ClientSession(trace_configs=[trace_config], middlewares=(middleware,)) as session:
async with session.get("http://something") as response:
print(response.status)

if __name__ == "__main__":
asyncio.run(main())
```

For an endpoint that always returns 500 we will see the following:
```
on request start
middleware start
attempt 0
attempt 1
attempt 2
middleware end
on request end
500
```

Which means if I use OpenTelemetry instrumentation, I would see a single request where I would expect to see 3.
Things become even more complicated when you take the [automatic retry for idempotent methods](https://github.com/aio-libs/aiohttp/issues/7297) into account.
At that point I can have up to **6 attempts** to send the request, but tracing-wise, I would only see one which (at least for me) is completly unexpected.

Is this a bug or the intended behavior? I would love to see some documentation about the way client middlewares should be integrated with other features of aiohttp.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.