psf / psf/requests

Address deleting cookies when domain-less defaults are used.

Open
#4,425 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
54.3k
Forks
10.4k
Avg merge
16h 43m
Merged PRs (30d)
3

Description

I found a few variations of this "implementation detail" in other reports, but I believe this particular use case is worth addressing.

If a URL is retrieved with either a dict or unqualified cookie jar, the deletions are ignored.

# use a dict
resp = requests.get(url, cookies={'cookie1': 'value'},)

or

# use a cookiejar
session1 = requests.Session()
session1.cookies = requests.cookies.RequestsCookieJar()
session1.cookies.set('foo', 'bar')
resp1 = requests.get(url)

This can create endless redirect loops if the server's response to a resource/url combination is to set a "delete cookie", which requests can not handle. Aside from failing tests and puzzled developers, this can trigger abuse flags with 3rd party api services.

Expected Result

If the server sends a cookie deletion value, it should be respected and not sent to the server.

Actual Result

passing a 'cookies=" dict to requests.get() creates a wrapped session with the default cookiejar. the current implementation just deletes any matching cookie for the domain, which fails because the cookie was set as a default (not for that domain).

Potential Way To Fix

A potential fix is to extend the RequestsCookieJar to use a blacklist when domain-less defaults are provided.

When a SetCookie deletion is encountered and the cookiejar contains domain-less cookies:

  • delete the domain cookie (if it exists)
  • and populate the blacklist.

When a request is made, filter the defaults against active blacklist values.

When a SetCookie update is made, remove it from the blacklist (or update the blacklist's to note it as invalid until the cookie's expiry date)

Reproduction Steps

import requests

# cookies not unset
session1 = requests.Session()
session1.cookies = requests.cookies.RequestsCookieJar()
session1.cookies.set('foo', 'bar')

r_set = session1.get("http://httpbin.org/cookies/set?bar=foo")
print(r_set.json())
r_del = session1.get("http://httpbin.org/cookies/delete?foo=bar")
print(r_del.json())
r_check = session1.get("http://httpbin.org/cookies")
print(r_check.json())

# cookies unset because domain matches
session2 = requests.Session()
session2.cookies = requests.cookies.RequestsCookieJar()
session2.cookies.set('foo', 'bar', domain='httpbin.org', path='/')

rr_set = session2.get("http://httpbin.org/cookies/set?bar=foo")
print(rr_set.json())
rr_del = session2.get("http://httpbin.org/cookies/delete?foo=bar")
print(rr_del.json())
rr_check = session2.get("http://httpbin.org/cookies")
print(rr_check.json())

System Information

"version": "2.18.4"

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the reproduction steps using requests.Session and RequestsCookieJar, comparing domain-less cookies with cookies set for httpbin.org. Trace how requests.get handles Set-Cookie deletion and verify the completed behavior by checking that the final /cookies response no longer includes the deleted cookie in both cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.