psf / psf/requests

Urllib3 >1.26.x | Introduces default User-Agent header

Open
#5,671 2 comments 0 reactions 1 assignee View on GitHub

@sigmavirus24 is already working on this.

Since Dec 12, 2020.

Dominant language
Python
Stars
54.3k
Forks
10.4k
Avg merge
16h 43m
Merged PRs (30d)
3

Description

With pyrequest pullrequest https://github.com/psf/requests/commit/03957eb1c2b9a1e5e6d61f5e930d7c5ed7cfe853 v2.25.0 urllib3 is updated to v1.26.0. with this version bump urllib3 has changed default behavior in regards to header "User-Agent" : https://pypi.org/project/urllib3/

urllib3 1.26.0

Added default User-Agent header to every request (Pull #1750)
Added urllib3.util.SKIP_HEADER for skipping User-Agent, Accept-Encoding, and Host headers from being automatically emitted with requests (Pull #2018)

Urllib3 now introduces default User-Agent header when no user-agent header is set. Urlib3 also introduces the function "urllib3.util.SKIP_HEADER" as a means to deactivate default user-agent header.

Expected Result

Pyrequest should adopt easy means of deactivating "User-Agent" headers by utilizing the "urllib3.util.SKIP_HEADER" function

Actual Result

Default "User-agent" header is enforced when no header= is specified.
by using headers={"User-Agent": none} no longer deactivate default "user-agent" while using headers={"User-Agent": ""} will deactivate "user-agent" header

Reproduction Steps

import requests

req = Request('POST', url=BASE_URL + endpoint, data=params).prepare()  

---> Deafult "user-agent" is enforced "User-Agent: python-urllib3/1.26.2"

req = Request('POST', url=BASE_URL + endpoint, headers={"User-Agent": None}, data=params).prepare()

---> Does not deactivate "User-Agent"

req = Request('POST', url=BASE_URL + endpoint, headers={"User-Agent": ""}, data=params).prepare()

Deactivates "User-Agent" succesfully


## System Information

    $ python -m requests.help

```

This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c).

Python 3.8, Docker

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.