aio-libs / aio-libs/aiohttp

logging of raw request/response

Đang mở
#6,884 10 bình luận 9 reaction 0 người được giao Xem trên GitHub
enhancement
Ngôn ngữ chính
Python
Star
16.5k
Fork
2.4k
Merge trung bình
20 giờ 10 phút
Pull request đã merge (30 ngày)
221

Mô tả

### Is your feature request related to a problem?

I have scenarios where I need to be able to see the raw request/response for a given HTTP interaction. For example, I'm currently getting a 401 unauthorized back from Twilio, and I need to see the raw request/response to diagnose why it's failing because I know the credentials are correct (I know this because I had to convert some lua code to python and the code and credentials were working without a problem in lua).

This is also useful when trying to prove to an API vendor that they're doing something wrong like sending an html content-type but the content itself is obviously json, etc.

### Describe the solution you'd like

I'd like a simple, clean, supported way to opt-in to raw logging of requests/responses. This would probably be cleanest by allowing an optional logging callback in ClientSession's __init__() as well as the individual request methods, for example:

```
async with aiohttp.ClientSession( rawlog = lambda s: log.debug( s )) as session:
...
```

or if I didn't want session-wide logging:

```
async with aiohttp.ClientSession() as session:
async with session.get( url, rawlog = lambda s: log.debug( s )) as rsp:
```

Alternatively, you could do something like this:

```
aiohttp.log.client_logger.log( 1, '....' )
```

This would require users to explicitly enable a logging level of 1 ( which is less even than logging.DEBUG ) in order to opt-in to this behavior.

### Describe alternatives you've considered

I created the following abomination to meet my current needs, but it is likely to break everytime the developers sneeze:

```
# aiohttp_logging.py

from __future__ import annotations

import aiohttp.client_reqrep
import logging
from typing import Any, Tuple, TYPE_CHECKING

if TYPE_CHECKING:
from aiohttp.connector import Connection

logger = logging.getLogger( __name__ )

OriginalClientRequest_send = aiohttp.client_reqrep.ClientRequest.send
OriginalStreamWriter = aiohttp.client_reqrep.StreamWriter
OriginalClientResponse_start = aiohttp.client_reqrep.ClientResponse.start
OriginalClientResponse_read = aiohttp.client_reqrep.ClientResponse.read

async def LoggingClientRequest_send(self, conn: Connection) -> aiohttp.client_reqrep.ClientResponse:
logger.debug( f'requesting {self.method} {self.original_url}' )
return await OriginalClientRequest_send( self, conn )

class LoggingStreamWriter( OriginalStreamWriter ):
def _write( self, chunk: bytes ) -> None:
#log = logger.getChild( 'LoggingStreamWriter._write' )
super()._write( chunk )
for line in chunk.decode( 'cp437' ).split( '\n' )[:-1]: # -1 because an extra blank lines gets shown otherwise...
logger.debug( f'C>{line}' )

class LoggingProtocol:
def __init__( self, protocol: 'Protocol' ) -> None:
self.protocol = protocol

async def read( self ) -> Tuple[Any,Any]:
#log = logger.getChild( 'LoggingProtocol.read' )
msg, payload = await self.protocol.read()
logger.debug( f'S>HTTP/{msg.version.major}.{msg.version.minor} {msg.code} {msg.reason}' )
for name, value in msg.raw_headers:
logger.debug( f'S>{name.decode("cp437")}: {value.decode("cp437")}' )
logger.debug( 'S>' ) # show blank line between headers and body
return msg, payload

async def LoggingClientResponse_start( self, connection: Connection ) -> aiohttp.client_reqrep.ClientResponse:
#log = logger.getChild( 'LoggingClientResponse_start' )
orig_protocol = connection.protocol
try:
connection._protocol = LoggingProtocol( orig_protocol )
return await OriginalClientResponse_start( self, connection )
finally:
connection._protocol = orig_protocol

async def LoggingClientResponse_read( self ) -> bytes:
#log = logger.getChild( 'LoggingClientResponse_read' )
data = await OriginalClientResponse_read( self )
for line in data.split( b'\n' ):
logger.debug( f'S>{line.decode("cp437").rstrip()}' )
return data

def monkey_patch() -> None:
assert aiohttp.client_reqrep.StreamWriter == OriginalStreamWriter, 'already monkey-patched'
aiohttp.client_reqrep.ClientRequest.send = LoggingClientRequest_send
aiohttp.client_reqrep.StreamWriter = LoggingStreamWriter
aiohttp.client_reqrep.ClientResponse.start = LoggingClientResponse_start
aiohttp.client_reqrep.ClientResponse.read = LoggingClientResponse_read
```

### Related component

Client

### Additional context

sample output from a test performed against a little netgear switch on my network:
```
DEBUG:ace.incpy.aiohttp_logging:requesting GET http://172.16.42.2/
DEBUG:ace.incpy.aiohttp_logging:C>GET / HTTP/1.1
DEBUG:ace.incpy.aiohttp_logging:C>Host: 172.16.42.2
DEBUG:ace.incpy.aiohttp_logging:C>Accept: */*
DEBUG:ace.incpy.aiohttp_logging:C>Accept-Encoding: gzip, deflate
DEBUG:ace.incpy.aiohttp_logging:C>User-Agent: Python/3.9 aiohttp/3.8.1
DEBUG:ace.incpy.aiohttp_logging:C>
DEBUG:ace.incpy.aiohttp_logging:S>HTTP/1.1 200 OK
DEBUG:ace.incpy.aiohttp_logging:S>Connection: close
DEBUG:ace.incpy.aiohttp_logging:S>X-Frame-Options: SAMEORIGIN
DEBUG:ace.incpy.aiohttp_logging:S>X-XSS-Protection: 1; mode=block
DEBUG:ace.incpy.aiohttp_logging:S>X-Content-Type-Options: nosniff
DEBUG:ace.incpy.aiohttp_logging:S>Content-Type: text/html
DEBUG:ace.incpy.aiohttp_logging:S>Cache-Control: no-cache
DEBUG:ace.incpy.aiohttp_logging:S>Expires: -1
DEBUG:ace.incpy.aiohttp_logging:S>
DEBUG:ace.incpy.aiohttp_logging:S>
DEBUG:ace.incpy.aiohttp_logging:S>
(snip)
DEBUG:ace.incpy.aiohttp_logging:S>
DEBUG:ace.incpy.aiohttp_logging:S>
```

### Code of Conduct

- [X] I agree to follow the aio-libs Code of Conduct

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.