psf / psf/requests

Chardet is used, when it is available, not when `[use-chardet-on-py3]`-extra is installed

Open
#7,223 11 comments 4 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
54.3k
Forks
10.4k
Avg merge
16h 43m
Merged PRs (30d)
3

Description

First: IMHO this is related to but not a duplicate of #5871 #7222 #7219

requests tries to use chardet when it can be imported. However, IMHO it should only use it, if requests was installed with the [use-chardet-on-py3]-extra.

As reported in #7219, requests shows a warning, when the version of chardet does not fulfill the requirements. However, package managers like uv won't respect the version requirement of the dependency of requests on chardet, when the extra is not installed (see https://github.com/astral-sh/uv/issues/18179). If another project dependency depends on chardet, it will be installed, but the version of chardet might not fulfill the requirement of requests.

The same applies to charset_normalizer or any other optional dependency.

Expected Result

chardet should not be used, when requests is not installed with [use-chardet-on-py3]-extra.

Actual Result

requests tries to use chardet and generates a log message complaining about an incompatible version.

Reproduction Steps

uv init
# I use fixed versions to make the example more reproducible:
uv add tox==4.44.0 requests==2.32.5

cat > test.py <<EOF
import logging

LOGGER = logging.getLogger()
LOGGER.setLevel(logging.DEBUG)

import chardet
import requests
import tox

print("tox", tox.__version__)
print("requests", requests.__version__)
print("chardet", chardet.__version__)
EOF
uv run test.py
Output:
.venv/lib/python3.12/site-packages/requests/__init__.py:113: RequestsDependencyWarning: urllib3 (2.6.3) or chardet (6.0.0.post1)/charset_normalizer (3.4.4) doesn't match a supported version!
  warnings.warn(
tox 4.44.0
requests 2.32.5
chardet 6.0.0.post1

System Information

$ python -m requests.help
{
  "chardet": {
    "version": "6.0.0.post1"
  },
  "charset_normalizer": {
    "version": "3.4.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.11"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.12.3"
  },
  "platform": {
    "release": "6.17.0-14-generic",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.32.5"
  },
  "system_ssl": {
    "version": "300000d0"
  },
  "urllib3": {
    "version": "2.6.3"
  },
  "using_charset_normalizer": false,
  "using_pyopenssl": false
}

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the supplied uv reproduction and inspect requests/init.py, where the warning appears. Trace how optional chardet and charset_normalizer availability is detected and compare that with the use-chardet-on-py3 extra. Done means the reproduction no longer uses or warns about chardet unless that extra is installed, while the existing optional-dependency behavior remains covered.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.