Incorrect cookie handling: third level domain incorrectly uses second level domain cookies
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 54.3k
- Forks
- 10.5k
- Avg merge
- 16h 43m
- Merged PRs (30d)
- 3
Description
When visiting (logging in to) zh.wikisource.org, cookies for the following domain are set:
- .wikisource.org
If one visits wikisource.org afterwards, cookies for the following domain are set:
- wikisource.org
After i.e. logout from zh.wikisource.org, zh.wikisource.org sends cookie invalidation request for .wikisource.org. Requests invalidates them correctly, leaves wikisource.org cookies untouched correctly.
But for further communication with zh.wikisource.org, requests incorrectly use remaining wikisource.org cookie. Which is rejected by zh.wikisource.org and followed by cookie invalidation request for .wikisource.org.
Expected Result
It looks like the cookie handling library used by requests is doing something wrong (at least according to the RFCs), as it should not send the host-only cookie for wikisource.org to zh.wikisource.org. If it's using RFC 2109, "zh.wikisource.org" does not domain-match the cookie for "wikisource.org" because "wikisource.org" doesn't begin with a dot. If it's using RFC 6265, the domainless cookie for "wikisource.org" should have had the host-only-flag set meaning it should not be sent to "zh.wikisource.org". OTOH, it's possible it's being bug-compatible with browsers (RFC 6265 even notes that such a bug exists/existed in some agents in § 4.1.2.3).
Actual Result
As written above, requests use wikisource.org cookie for zh.wikisource.org. Which is rejected by zh.wikisource.org and followed by cookie invalidation request for .wikisource.org". That is fulfilled (no cookies for .wikisource.org are set), but requests still tries to push cookies for wikisource.org to zh.wikisource.org. So the actual result is endless loop (https://travis-ci.org/github/wikimedia/pywikibot/jobs/669558038#L13763).
Reproduction Steps
Simple:
import pywikibot
s=pywikibot.Site('zh', 'wikisource')
print(s.hostname()) # zh.wikisource.org
s.login()
s2=pywikibot.Site('mul', 'wikisource')
print(s2.hostname()) # wikisource.org
s2.login()
s.logout()
s.login()
# endless loop here
Elaborate:
import requests
s=requests.Session()
def login(url):
while True:
# login token
r1=s.post(url=url, data={'action':'query','meta':'tokens','type':'login','format':'json'})
logintoken = r1.json()['query']['tokens']['logintoken']
# login (test account)
r2=s.post(url=url, data={'action':'clientlogin','loginreturnurl':'https://example.com','logintoken':logintoken,'username':'Test20200402','password':'popokatepetl','format':'json'})
print(r2.json())
if r2.json().get('error') and r2.json()['error'].get('code') == 'badtoken':
continue
else:
break
#print(s.cookies)
def logout(url):
# logout token
r1=s.post(url=url, data={'action':'query','meta':'tokens','type':'csrf','format':'json'})
logouttoken = r1.json()['query']['tokens']['csrftoken']
# logout
r2=s.post(url=url, data={'action':'logout','token':logouttoken,'format':'json'})
print(r2.json())
#print(s.cookies)
url1 = 'https://zh.wikisource.org/w/api.php'
url2='https://wikisource.org/w/api.php'
login(url1)
login(url2)
logout(url1)
login(url1)
# endless loop here
System Information
$ python -m requests.help
Reproduced on many PCs with many configurations and also on Travis-CI. Here is one:
{
"chardet": {
"version": "3.0.4"
},
"cryptography": {
"version": ""
},
"idna": {
"version": "2.6"
},
"implementation": {
"name": "CPython",
"version": "3.6.9"
},
"platform": {
"release": "5.4.18",
"system": "Linux"
},
"pyOpenSSL": {
"openssl_version": "",
"version": null
},
"requests": {
"version": "2.23.0"
},
"system_ssl": {
"version": "1010104f"
},
"urllib3": {
"version": "1.22"
},
"using_pyopenssl": false
}
Links:
Reported and described in more detail in: https://phabricator.wikimedia.org/T224712
Reproducible immediately in: https://repl.it/repls/HarmfulBiodegradableExperiments
Pastebin (just in case): https://pastebin.com/7d7Dn9p1
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the requests.Session reproduction using the two wikisource.org URLs and inspect how cookies are selected for each request. Compare the behavior with the expected host-only and domain-matching rules described in the issue. Done means the wikisource.org cookie is not sent to zh.wikisource.org and the repeated login loop no longer occurs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100