psf / psf/requests

Cookies are lost on 307 redirects

Open
#5,709 1 comment 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
54.3k
Forks
10.4k
Avg merge
16h 43m
Merged PRs (30d)
3

Description

When I submit a request, I get a 307 response and a cookie. But when the request is repeated, the cookie is not added.

Added request logging:

try:
    import http.client as http_client
except ImportError:
    # Python 2
    import httplib as http_client
http_client.HTTPConnection.debuglevel = 1

# You must initialize logging, otherwise you'll not see debug output.
import logging
logging.basicConfig()
logging.getLogger().setLevel(logging.DEBUG)
requests_log = logging.getLogger("requests.packages.urllib3")
requests_log.setLevel(logging.DEBUG)
requests_log.propagate = True

Result (first 2 requests):

DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): www.dns-shop.ru:443
send: b'GET /product/4d664a0d90d61b80/processor-amd-ryzen-7-3700x-oem/ HTTP/1.1\r\nHost: www.dns-shop.ru\r\nUser-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:84.0) Gecko/20100101 Firefox/84.0\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nConnection: keep-alive\r\n\r\n'
DEBUG:urllib3.connectionpool:https://www.dns-shop.ru:443 "GET /product/4d664a0d90d61b80/processor-amd-ryzen-7-3700x-oem/ HTTP/1.1" 307 0
reply: 'HTTP/1.1 307 TemporaryRedirect\r\n'
header: Server: Variti/0.9.3a
header: Date: Fry, 25 Dec 2020 14:3753 GMT
header: Set-Cookie: ipp_uid_tst=1608907073940/2LhuRHu-NT--na0AX4hLKg; Expires=; Domain=; Path=/
header: Location: /product/4d664a0d90d61b80/processor-amd-ryzen-7-3700x-oem/
header: Connection: keep-alive
header: Keep-Alive: timeout=60
header: Content-Length: 0

send: b'GET /product/4d664a0d90d61b80/processor-amd-ryzen-7-3700x-oem/ HTTP/1.1\r\nHost: www.dns-shop.ru\r\nUser-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:84.0) Gecko/20100101 Firefox/84.0\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nConnection: keep-alive\r\n\r\n'
DEBUG:urllib3.connectionpool:https://www.dns-shop.ru:443 "GET /product/4d664a0d90d61b80/processor-amd-ryzen-7-3700x-oem/ HTTP/1.1" 307 0
reply: 'HTTP/1.1 307 TemporaryRedirect\r\n'
header: Server: Variti/0.9.3a
header: Date: Fry, 25 Dec 2020 14:3754 GMT
header: Set-Cookie: ipp_uid_tst=1608907074028/oOg1XsGU12ZE8PkUludY5g; Expires=; Domain=; Path=/
header: Location: /product/4d664a0d90d61b80/processor-amd-ryzen-7-3700x-oem/
header: Connection: keep-alive
header: Keep-Alive: timeout=60
header: Content-Length: 0

Expected Result

HTTP 200, OK

Actual Result

Traceback (most recent call last):
  File "C:/Users/ipetrash/Projects/SimplePyScripts/html_parsing/www_dns_shop_ru/get_price.py", line 17, in <module>
    rs = requests.get('https://www.dns-shop.ru/')
  File "C:\Users\ipetrash\Anaconda3\lib\site-packages\requests\api.py", line 76, in get
    return request('get', url, params=params, **kwargs)
  File "C:\Users\ipetrash\Anaconda3\lib\site-packages\requests\api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "C:\Users\ipetrash\Anaconda3\lib\site-packages\requests\sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Users\ipetrash\Anaconda3\lib\site-packages\requests\sessions.py", line 677, in send
    history = [resp for resp in gen]
  File "C:\Users\ipetrash\Anaconda3\lib\site-packages\requests\sessions.py", line 677, in <listcomp>
    history = [resp for resp in gen]
  File "C:\Users\ipetrash\Anaconda3\lib\site-packages\requests\sessions.py", line 166, in resolve_redirects
    raise TooManyRedirects('Exceeded {} redirects.'.format(self.max_redirects), response=resp)
requests.exceptions.TooManyRedirects: Exceeded 30 redirects.

Reproduction Steps

import requests

session = requests.session()
rs = session.get('https://www.dns-shop.ru/')
print(rs)

System Information

$ python -m requests.help
{                                   
  "chardet": {                      
    "version": "3.0.4"              
  },                                
  "cryptography": {                 
    "version": "2.7"                
  },                                
  "idna": {                         
    "version": "2.8"                
  },                                
  "implementation": {               
    "name": "CPython",              
    "version": "3.7.3"              
  },                                
  "platform": {                     
    "release": "10",                
    "system": "Windows"             
  },                                
  "pyOpenSSL": {                    
    "openssl_version": "1010103f",  
    "version": "17.2.0"             
  },                                
  "requests": {                     
    "version": "2.25.1"             
  },                                
  "system_ssl": {                   
    "version": "1010108f"           
  },                                
  "urllib3": {                      
    "version": "1.24.2"             
  },                                
  "using_pyopenssl": true           
}                                   

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at requests.sessions.Session.resolve_redirects, reached by the supplied session.get reproduction, and inspect how the 307 response's Set-Cookie header is handled before the repeated request. Reproduce against the logged dns-shop.ru response and verify that the cookie is included on the redirected request and the redirect loop no longer occurs.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
networking
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.