aio-libs / aio-libs/aiohttp

Content-Encoding with multiple codings is not decoded

Offen
#13,364 6 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
bug
Vorherrschende Sprache
Python
Sterne
16.5k
Forks
2.4k
Ø Merge
17 Std. 22 Min.
Gemergte PRs (30 T.)
212

Beschreibung

### Describe the bug

RFC 9110 §8.4 allows `Content-Encoding` to list several codings. The client must decode them in reverse order. When a server sends a body compressed twice and the header `Content-Encoding: gzip,gzip`, aiohttp does not decode the body. It passes the raw gzip bytes to `resp.json()` (and `resp.text()`), which then tries to UTF-8-decode the gzip magic and fails.

### To Reproduce

Two files. No TLS, no proxy.

`server.py` (stdlib only) always answers with a doubly-gzipped JSON body and declares `Content-Encoding: gzip,gzip`:

```python
#!/usr/bin/env python3
import gzip
import json
import socket

body = gzip.compress(gzip.compress(json.dumps({"hello": "world"}).encode()))
srv = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
srv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
srv.bind(("127.0.0.1", 8080))
srv.listen()
while True:
conn, _ = srv.accept()
conn.recv(65536)
head = "\r\n".join((
"HTTP/1.1 200 OK",
"Content-Type: application/json",
"Content-Encoding: gzip,gzip",
f"Content-Length: {len(body)}",
"Connection: close",
"",
"",
)).encode()
conn.sendall(head + body)
conn.close()
```

`client.py`:

```python
#!/usr/bin/env python3
import asyncio
import aiohttp

async def fetch():
async with aiohttp.ClientSession() as session:
async with session.get("http://127.0.0.1:8080/") as resp:
print(await resp.json())

asyncio.run(fetch())
```

Run:

```
./server.py &
uv run --with aiohttp==3.14.3 ./client.py
```

### Expected behavior

aiohttp decodes each coding listed in `Content-Encoding`, in reverse order (here: gunzip twice), and `resp.json()` returns `{"hello": "world"}`.

### Logs/tracebacks

```python-traceback
$ ./server.py &
$ uv run --with aiohttp==3.14.3 ./client.py
Installed 9 packages in 8ms
Traceback (most recent call last):
File "/tmp/tg/./client.py", line 12, in
asyncio.run(fetch())
~~~~~~~~~~~^^^^^^^^^
File "/home/philip/.local/share/uv/python/cpython-3.13.11-linux-x86_64-gnu/lib/python3.13/asyncio/runners.py", line 195, in run
return runner.run(main)
~~~~~~~~~~^^^^^^
File "/home/philip/.local/share/uv/python/cpython-3.13.11-linux-x86_64-gnu/lib/python3.13/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
File "/home/philip/.local/share/uv/python/cpython-3.13.11-linux-x86_64-gnu/lib/python3.13/asyncio/base_events.py", line 725, in run_until_complete
return future.result()
~~~~~~~~~~~~~^^
File "/tmp/tg/./client.py", line 9, in fetch
print(await resp.json())
^^^^^^^^^^^^^^^^^
File "/home/philip/.cache/uv/archive-v0/6kdapgyXRUIN9ElB/lib/python3.13/site-packages/aiohttp/client_reqrep.py", line 795, in json
return loads(stripped.decode(encoding))
~~~~~~~~~~~~~~~^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte
```

### Python Version

```console
$ uv run --with aiohttp==3.14.3 python --version
Python 3.13.11
```

### aiohttp Version

```console
$ uv run --with aiohttp==3.14.3 python -m pip show aiohttp
Name: aiohttp
Version: 3.14.3
Summary: Async http client/server framework (asyncio)
Home-page: https://github.com/aio-libs/aiohttp
Author:
Author-email:
License: Apache-2.0 AND MIT
Location: /home/philip/.cache/uv/archive-v0/6kdapgyXRUIN9ElB/lib/python3.13/site-packages
Requires: aiohappyeyeballs, aiosignal, attrs, frozenlist, multidict, propcache, yarl
Required-by:
```

### multidict Version

```console
$ uv run --with aiohttp==3.14.3 python -m pip show multidict
Name: multidict
Version: 6.7.1
Summary: multidict implementation
Home-page: https://github.com/aio-libs/multidict
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache License 2.0
Location: /home/philip/.cache/uv/archive-v0/6kdapgyXRUIN9ElB/lib/python3.13/site-packages
Requires:
Required-by: aiohttp, yarl
```

### propcache Version

```console
$ uv run --with aiohttp==3.14.3 python -m pip show propcache
Name: propcache
Version: 0.5.2
Summary: Accelerated property cache
Home-page: https://github.com/aio-libs/propcache
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache-2.0
Location: /home/philip/.cache/uv/archive-v0/6kdapgyXRUIN9ElB/lib/python3.13/site-packages
Requires:
Required-by: aiohttp, yarl
```

### yarl Version

```console
$ uv run --with aiohttp==3.14.3 python -m pip show yarl
Name: yarl
Version: 1.24.5
Summary: Yet another URL library
Home-page: https://github.com/aio-libs/yarl
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache-2.0
Location: /home/philip/.cache/uv/archive-v0/6kdapgyXRUIN9ElB/lib/python3.13/site-packages
Requires: idna, multidict, propcache
Required-by: aiohttp
```

### OS

Arch Linux

### Related component

Client

### Additional context

Suspected cause: `HttpParser.parse_headers()` in `aiohttp/http_parser.py:632-635` matches the whole header against the set `{"gzip", "deflate", "br", "zstd"}`. A value like `gzip,gzip` does not match, so `encoding` stays `None` (line 608), the `DeflateBuffer` wrapper is
skipped (line 882), and raw bytes pass through. Multi-coding `Content-Encoding` is simply not implemented.

### Code of Conduct

- [x] I agree to follow the aio-libs Code of Conduct

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.