aio-libs / aio-libs/aiohttp

Wrong URL parsing in redirect

Đang mở
#6,626 12 bình luận 0 reaction 0 người được giao Xem trên GitHub
bug
Ngôn ngữ chính
Python
Star
16.5k
Fork
2.4k
Merge trung bình
17 giờ 22 phút
Pull request đã merge (30 ngày)
212

Mô tả

### Describe the bug

I'm scrapping the marvel fandom site using tasks of asyncio.
Every time that I try to get the HTML document from a URL that contains the '&' symbol I have the same issue:

For example with: https://marvel.fandom.com/wiki/Amazing_Spider-Man_&_Silk:_The_Spider(fly)_Effect_Vol_1

This link redirects to another link where the & symbol is replaced with the % encoding (& = %26). So the actual URL of the page is https://marvel.fandom.com/wiki/Amazing_Spider-Man_%26_Silk:_The_Spider(fly)_Effect_Vol_1

Then I run the following code and I get an error

![image](https://user-images.githubusercontent.com/18238415/154723378-0ec00927-6a4a-4a32-9350-9c98241a6337.png)

To catch the exception I made the following:

![image](https://user-images.githubusercontent.com/18238415/154724157-fa51c738-f2c1-4bea-ac90-460443c36a34.png)

And as you can see the code redirects many times until max_amount_redirects is reached. The problem is that the actual URL obtained in the redirection ('Location': 'https://marvel.fandom.com/wiki/Amazing_Spider-Man_%26_Silk:_The_Spider(fly)_Effect_Vol_1') apparently is again parsed with % code and %26 is replaced by &.

The same error occurs if I use https://marvel.fandom.com/wiki/Amazing_Spider-Man_%26_Silk:_The_Spider(fly)_Effect_Vol_1 in the first time, %26 is replaced by & and redirects until limit is reached. To make it work I had to replace & with %%26 so it parses to a literal "%26" in the url.

### To Reproduce

![image](https://user-images.githubusercontent.com/18238415/154723378-0ec00927-6a4a-4a32-9350-9c98241a6337.png)

### Expected behavior

Redirects correctly

### Logs/tracebacks

```python-traceback
No applicable
```

### Python Version

```console
$ python --version

Python 3.10.1
```

### aiohttp Version

```console
$ python -m pip show aiohttp

Name: aiohttp
Version: 3.8.1
Summary: Async http client/server framework (asyncio)
Home-page: https://github.com/aio-libs/aiohttp
Author: None
Author-email: None
License: Apache 2
```

### multidict Version

```console
$ python -m pip show multidict

Name: multidict
Version: 6.0.2
Summary: multidict implementation
Home-page: https://github.com/aio-libs/multidict
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache 2
```

### yarl Version

```console
$ python -m pip show yarl

Name: yarl
Version: 1.7.2
Summary: Yet another URL library
Home-page: https://github.com/aio-libs/yarl/
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache 2
```

### OS

Manjaro, Linux

### Related component

Client

### Additional context

_No response_

### Code of Conduct

- [X] I agree to follow the aio-libs Code of Conduct

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.