psf / psf/requests

`uri` field of digest authentication incorrectly filled when the URL contains semicolons in path

Open
#6,990 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
54.3k
Forks
10.5k
Avg merge
16h 43m
Merged PRs (30d)
3

Description

When there is a semicolon (;) in the request URL path, and when doing HTTP Digest Authentication, requests incorrectly fills the uri field of the Authorization header by ignoring the rest of the path.

As per RFC 7616, the uri field contains the "Effective Request URI", which should contain the same path we are trying to access.

Expected Result

The uri field should contain the same path as the request when it contains semicolons.

Actual Result

The uri field removes everything from the first semicolon it finds in the path, up to the parameters.

Reproduction Steps

The following example is where I found this problem: this tries to add 2 releases to a MusicBrainz collection.
The API uses a semicolon as a separator for adding multiple releases at once.

The following will return a 401 since the collection is private, but since the problem is in the uri field, it is not a problem to reproduce the bug.

import requests
from requests.auth import HTTPDigestAuth

res = requests.put("https://musicbrainz.org/ws/2/collection/53f4a001-eb45-4b72-9ec5-41109e88710d/releases/7dc2cfbd-5bd8-4ebc-b20b-4344985431da;38347564-5ef3-46dd-ad87-fe6d6f1e7b19?fmt=json&client=manual-python-requests-test", auth=HTTPDigestAuth("username", "password"))

res.request.headers['Authorization']
>>> [...] uri="/ws/2/collection/53f4a001-eb45-4b72-9ec5-41109e88710d/releases/7dc2cfbd-5bd8-4ebc-b20b-4344985431da?fmt=json&client=manual-python-requests-test" [...]

Only the first release has been kept! The query parameters are however untouched, so this is not a simple "I see semicolon, I ignore everything after".

Also, doing the same request with curl works as expected, as it correctly fills the uri field:

curl -X PUT "https://musicbrainz.org/ws/2/collection/53f4a001-eb45-4b72-9ec5-41109e88710d/releases/7dc2cfbd-5bd8-4ebc-b20b-4344985431da;38347564-5ef3-46dd-ad87-fe6d6f1e7b19?fmt=json&client=manual-curl-test" --digest -u "username:password" -v

# [...] A lot of lines
> [...] uri="/ws/2/collection/53f4a001-eb45-4b72-9ec5-41109e88710d/releases/7dc2cfbd-5bd8-4ebc-b20b-4344985431da;38347564-5ef3-46dd-ad87-fe6d6f1e7b19?fmt=json&client=manual-curl-test" [...]

Also, the request works with requests if I manually escape the semicolons as %3B, but one shouldn't have to.

System Information

$ python -m requests.help
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "3.4.2"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.10"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.13.5"
  },
  "platform": {
    "release": "6.15.6",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.32.4"
  },
  "system_ssl": {
    "version": "30200040"
  },
  "urllib3": {
    "version": "2.5.0"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the HTTPDigestAuth entry point and reproduce the provided PUT request with a semicolon in its path. Compare the generated Authorization header with the effective request path; done means the digest uri preserves the full path, including text after the semicolon, while leaving the query parameters intact.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
networking
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.