Merging of default HTTP headers with specified headers breaks the defined ordering
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 54.3k
- Forks
- 10.4k
- Avg merge
- 16h 43m
- Merged PRs (30d)
- 3
Description
Even if headers are defined in an OrderedDict, their order might change if they coincide with the default_headers set in utils.py, namely User-Agent, Accept-Encoding, Accept, and Connection.
Expected Result
The order of the headers should not change when passed as an OrderedDict. So for example:
import requests
from collections import OrderedDict
headers = OrderedDict([
("Accept", '1'),
("Accept-Encoding", '2'),
("User-Agent", '3'),
])
r = requests.get('http://www.example.com', headers=headers)
should result in the Accept header coming first, followed by Accept-Encoding and finally User-Agent.
Actual Result
The User-Agent comes first, followed by Accept-Encoding and then Accept
Reason
The default headers --- as set by requests.utils.default_headers() --- are merged with the user specified headers by the function requests.sessions.merge_settings. This function first takes the default headers and then updates that with the user specified headers. This ensures that user specified headers overwrite the default ones.
However, the update process does not change the order of the default headers. So when setting User-Agent after Accept, this will not be taken into account as both headers are already in the default headers and in reverse order.
A fix to merge_settings could be to take the user specified headers first, and then iterate over the default headers and add them in if their are misssing.
Workaround
If one wants exactly the headers specified, one can first create a Session and remove the default headers:
s = requests.Session()
s.headers = {}
r = s.get(url, headers=headers, ...)
Having a way to change the default headers on the requests module would be a welcome addition. So one could change, for example, the user agent in one place and have it affect all requests calls that follow.
Reproduction Steps
import requests
from collections import OrderedDict
headers = OrderedDict([
("Accept", '1'),
("Accept-Encoding", '2'),
("User-Agent", '3'),
])
r = requests.get('http://www.example.com', headers=headers)
System Information
$ python -m requests.help
{
"chardet": {
"version": "4.0.0"
},
"cryptography": {
"version": "3.3.1"
},
"idna": {
"version": "2.10"
},
"implementation": {
"name": "CPython",
"version": "3.8.5"
},
"platform": {
"release": "5.8.0-50-generic",
"system": "Linux"
},
"pyOpenSSL": {
"openssl_version": "1010109f",
"version": "20.0.1"
},
"requests": {
"version": "2.25.1"
},
"system_ssl": {
"version": "1010106f"
},
"urllib3": {
"version": "1.26.4"
},
"using_pyopenssl": true
}
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in requests.sessions.merge_settings and compare its header merge behavior with requests.utils.default_headers(). Reproduce the OrderedDict example and verify that user-specified header order is preserved while missing default headers remain available.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 38/100