replace `get_encoding_from_headers` default with `None`
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 54.3k
- Forks
- 10.5k
- Avg merge
- 16h 43m
- Merged PRs (30d)
- 3
Description
This issue a suggestion for handling partial elements of issues #1737 and #2086 (#1774 and others too). I'm bringing this up in a separate ticket because it is a specific fix for multiple tickets and I am willing to write a PR for it in requests3 and requests2. (i would have just gone and generated a PR, but I can't figure out what the actual master for requests3 actually is)
my proposal is to do the following (some of which have already been discussed as possibilities):
alter get_encoding_from_headers()
replace the default value of get_encoding_from_headers with None when "text" is in the content_type header. currently this is "ISO-8859-1"
add Response.detected_encoding attribute
extend build_response and Response to additionally stash the header decoding in response.detected_encoding. this will allow the detected value to remain on the response object if response.encoding is changed by the consumer.
add Response.text_encoding property
add a text_encoding property to Response , which offloads some of the logic now in text, and will return "ISO-8859-1"
alter Response.text to use the new .text_encoding attribute
update the text property to use the new .text_encoding attribute. this just ties together all the above.
Goals
The goal of these changes is to positively identify situations where no response encoding was declared by the server. The backwards incompatibility is that None will now be returned when inspecting response.encoding when there is no encoding for the response - however the effective encoding used to generate .text will be available in response.text_encoding .
This still maintains compliance with RFC 2616, because .text will still interpret the lack of a declared encoding as having the required default charset of ISO-8859-1. This simply allows developers to know when ISO-8859-1 is being used in a default or explicit manner.
This approach doesn't address all aspects of those issues or close those tickets, but it offers a solution to many of the needs and concerns with some minor changes.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with requests/utils.py at get_encoding_from_headers and requests/models.py around Response.text, then trace build_response and the Response attributes involved. Compare the proposal with issues #1737, #2086, and #1774. Done means distinguishing an absent declared encoding from an explicit one while preserving ISO-8859-1 as the effective default for text decoding.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100