aio-libs / aio-libs/aiohttp

On redirects, middle URL with ø char gets parsed wrongly - leading to a 404

Abierto
#10,047 9 comentarios 0 reacciones 1 asignado Reclamado por @bizzyvinci Ver en GitHub
bug
Lenguaje dominante
Python
Estrellas
16.5k
Forks
2.4k
Merge medio
17 h 22 min
PR fusionados (30 d)
212

Descripción

### Describe the bug

Hello,

If I try to fetch this URL using aiohttp `https://cornelius-k.dk/synsproeve/`, it will redirect, eventually leading to a 404 when trying to get `https://cornelius-k.dk/synspr\udcf8ve` at the end of the chain.

Looks like the `Location` header will be parsed wrongly from `b'https://cornelius-k.dk/synspr\xf8ve'` which I found in the `Response._raw_headers`.

### To Reproduce

Code block:
```
import aiohttp
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.97 Safari/537.36'
}
async def fetch_url(url):
async with aiohttp.ClientSession(headers=headers) as session:
async with session.get(url) as response:
for i in response.history:
print(i.url)
print(i._headers)
print(i._raw_headers)
return response.status
print(await fetch_url("https://cornelius-k.dk/synsproeve/"))
```
Final URL in the redirect chain will be `https://cornelius-k.dk/synspr�ve` instead of `https://cornelius-k.dk/synsprøve` and 404 will be yielded.

### Expected behavior

Parsing URL in the redirects correctly and fetching the correct final URL.

### Logs/tracebacks

```python-traceback
Output of the code block:

https://cornelius-k.dk/synsproeve/

((b'server', b'nginx'), (b'date', b'Tue, 26 Nov 2024 16:02:17 GMT'), (b'content-type', b'text/html'), (b'content-length', b'162'), (b'd-cache', b'from-cache'), (b'cache-control', b'no-cache, no-store, must-revalidate'), (b'expires', b'Thu, 01 Jan 1970 00:00:00 GMT'), (b'x-content-type-options', b'nosniff'), (b'strict-transport-security', b'max-age=31536000; preload'), (b'x-frame-options', b'SAMEORIGIN'), (b'content-security-policy', b"frame-ancestors 'self'"), (b'location', b'https://cornelius-k.dk/synsproeve'), (b'd-geo', b'US'))
https://cornelius-k.dk/synsproeve

((b'server', b'nginx'), (b'date', b'Tue, 26 Nov 2024 16:02:18 GMT'), (b'content-type', b'text/html'), (b'content-length', b'162'), (b'location', b'http://cornelius-k.dk/synspr%C3%B8ve'), (b'd-cache', b'from-cache'), (b'cache-control', b'no-cache, no-store, must-revalidate'), (b'expires', b'Thu, 01 Jan 1970 00:00:00 GMT'), (b'x-content-type-options', b'nosniff'), (b'strict-transport-security', b'max-age=31536000; preload'), (b'x-frame-options', b'SAMEORIGIN'), (b'content-security-policy', b"frame-ancestors 'self'"), (b'd-geo', b'US'))
http://cornelius-k.dk/synspr%C3%B8ve

((b'Server', b'nginx'), (b'Date', b'Tue, 26 Nov 2024 16:02:18 GMT'), (b'Content-Length', b'0'), (b'Connection', b'keep-alive'), (b'Cache-Control', b'no-cache, no-store, must-revalidate'), (b'Expires', b'Thu, 01 Jan 1970 00:00:00 GMT'), (b'X-Content-Type-Options', b'nosniff'), (b'X-Frame-Options', b'SAMEORIGIN'), (b'Content-Security-Policy', b"frame-ancestors 'self'"), (b'Location', b'https://cornelius-k.dk/synspr\xf8ve'), (b'D-Geo', b'US'))
(404, URL('https://cornelius-k.dk/synspr�ve'))
```

### Python Version

```console
3.9.20
```

### aiohttp Version

```console
3.11.7
```

### multidict Version

```console
6.1.0
```

### propcache Version

```console
0.2.0
```

### yarl Version

```console
1.17.1
```

### OS

macOS

### Related component

Client

### Additional context

_No response_

### Code of Conduct

- [X] I agree to follow the aio-libs Code of Conduct

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.