aio-libs / aio-libs/aiohttp

Content-Encoding with multiple codings is not decoded

Abierto
#13,364 6 comentarios 0 reacciones 0 asignados Ver en GitHub
bug
Lenguaje dominante
Python
Estrellas
16.5k
Forks
2.4k
Merge medio
17 h 22 min
PR fusionados (30 d)
212

Descripción

### Describe the bug

RFC 9110 §8.4 allows `Content-Encoding` to list several codings. The client must decode them in reverse order. When a server sends a body compressed twice and the header `Content-Encoding: gzip,gzip`, aiohttp does not decode the body. It passes the raw gzip bytes to `resp.json()` (and `resp.text()`), which then tries to UTF-8-decode the gzip magic and fails.

### To Reproduce

Two files. No TLS, no proxy.

`server.py` (stdlib only) always answers with a doubly-gzipped JSON body and declares `Content-Encoding: gzip,gzip`:

```python
#!/usr/bin/env python3
import gzip
import json
import socket

body = gzip.compress(gzip.compress(json.dumps({"hello": "world"}).encode()))
srv = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
srv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
srv.bind(("127.0.0.1", 8080))
srv.listen()
while True:
conn, _ = srv.accept()
conn.recv(65536)
head = "\r\n".join((
"HTTP/1.1 200 OK",
"Content-Type: application/json",
"Content-Encoding: gzip,gzip",
f"Content-Length: {len(body)}",
"Connection: close",
"",
"",
)).encode()
conn.sendall(head + body)
conn.close()
```

`client.py`:

```python
#!/usr/bin/env python3
import asyncio
import aiohttp

async def fetch():
async with aiohttp.ClientSession() as session:
async with session.get("http://127.0.0.1:8080/") as resp:
print(await resp.json())

asyncio.run(fetch())
```

Run:

```
./server.py &
uv run --with aiohttp==3.14.3 ./client.py
```

### Expected behavior

aiohttp decodes each coding listed in `Content-Encoding`, in reverse order (here: gunzip twice), and `resp.json()` returns `{"hello": "world"}`.

### Logs/tracebacks

```python-traceback
$ ./server.py &
$ uv run --with aiohttp==3.14.3 ./client.py
Installed 9 packages in 8ms
Traceback (most recent call last):
File "/tmp/tg/./client.py", line 12, in
asyncio.run(fetch())
~~~~~~~~~~~^^^^^^^^^
File "/home/philip/.local/share/uv/python/cpython-3.13.11-linux-x86_64-gnu/lib/python3.13/asyncio/runners.py", line 195, in run
return runner.run(main)
~~~~~~~~~~^^^^^^
File "/home/philip/.local/share/uv/python/cpython-3.13.11-linux-x86_64-gnu/lib/python3.13/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
File "/home/philip/.local/share/uv/python/cpython-3.13.11-linux-x86_64-gnu/lib/python3.13/asyncio/base_events.py", line 725, in run_until_complete
return future.result()
~~~~~~~~~~~~~^^
File "/tmp/tg/./client.py", line 9, in fetch
print(await resp.json())
^^^^^^^^^^^^^^^^^
File "/home/philip/.cache/uv/archive-v0/6kdapgyXRUIN9ElB/lib/python3.13/site-packages/aiohttp/client_reqrep.py", line 795, in json
return loads(stripped.decode(encoding))
~~~~~~~~~~~~~~~^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte
```

### Python Version

```console
$ uv run --with aiohttp==3.14.3 python --version
Python 3.13.11
```

### aiohttp Version

```console
$ uv run --with aiohttp==3.14.3 python -m pip show aiohttp
Name: aiohttp
Version: 3.14.3
Summary: Async http client/server framework (asyncio)
Home-page: https://github.com/aio-libs/aiohttp
Author:
Author-email:
License: Apache-2.0 AND MIT
Location: /home/philip/.cache/uv/archive-v0/6kdapgyXRUIN9ElB/lib/python3.13/site-packages
Requires: aiohappyeyeballs, aiosignal, attrs, frozenlist, multidict, propcache, yarl
Required-by:
```

### multidict Version

```console
$ uv run --with aiohttp==3.14.3 python -m pip show multidict
Name: multidict
Version: 6.7.1
Summary: multidict implementation
Home-page: https://github.com/aio-libs/multidict
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache License 2.0
Location: /home/philip/.cache/uv/archive-v0/6kdapgyXRUIN9ElB/lib/python3.13/site-packages
Requires:
Required-by: aiohttp, yarl
```

### propcache Version

```console
$ uv run --with aiohttp==3.14.3 python -m pip show propcache
Name: propcache
Version: 0.5.2
Summary: Accelerated property cache
Home-page: https://github.com/aio-libs/propcache
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache-2.0
Location: /home/philip/.cache/uv/archive-v0/6kdapgyXRUIN9ElB/lib/python3.13/site-packages
Requires:
Required-by: aiohttp, yarl
```

### yarl Version

```console
$ uv run --with aiohttp==3.14.3 python -m pip show yarl
Name: yarl
Version: 1.24.5
Summary: Yet another URL library
Home-page: https://github.com/aio-libs/yarl
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache-2.0
Location: /home/philip/.cache/uv/archive-v0/6kdapgyXRUIN9ElB/lib/python3.13/site-packages
Requires: idna, multidict, propcache
Required-by: aiohttp
```

### OS

Arch Linux

### Related component

Client

### Additional context

Suspected cause: `HttpParser.parse_headers()` in `aiohttp/http_parser.py:632-635` matches the whole header against the set `{"gzip", "deflate", "br", "zstd"}`. A value like `gzip,gzip` does not match, so `encoding` stays `None` (line 608), the `DeflateBuffer` wrapper is
skipped (line 882), and raw bytes pass through. Multi-coding `Content-Encoding` is simply not implemented.

### Code of Conduct

- [x] I agree to follow the aio-libs Code of Conduct

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.