python / python/typeshed

`urllib.request.Request.data` type erroring when using `decode`

Open
#11,470 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

status: deferred stubs: false positive
Dominant language
Python
Stars
5.1k
Forks
2.1k
Avg merge
1d 19h
Merged PRs (30d)
82

Description

When getting the data out of a urllib.request.Request object, mypy is erroring saying that decode is not supported even tho it is a bytes object (and does not error when at runtime).

$ cat t.py
from urllib.request import Request

req = Request("http://localhost:8080", data=b"req body")

if req.data:
    print(dir(req.data))
    print(type(req.data))
    print(req.data.decode())

$ mypy t.py
t.py:8: error: Item "Buffer" of "Buffer | SupportsRead[bytes] | Iterable[bytes]" has no attribute "decode"  [union-attr]
t.py:8: error: Item "SupportsRead[bytes]" of "Buffer | SupportsRead[bytes] | Iterable[bytes]" has no attribute "decode"  [union-attr]
t.py:8: error: Item "Iterable[bytes]" of "Buffer | SupportsRead[bytes] | Iterable[bytes]" has no attribute "decode"  [union-attr]
Found 3 errors in 1 file (checked 1 source file)

$ python t.py
[... 'decode', ...]
<class 'bytes'>
req body

$ python --version --version
Python 3.12.0 (main, Oct  2 2023, 20:56:14) [Clang 16.0.3 ]

$ mypy --version
mypy 1.8.0 (compiled: yes)

I believe PR #10225 had made this an error as it changed bytes to Buffer.

- ReadOnlyBuffer: TypeAlias = bytes
- ReadableBuffer: TypeAlias = ReadOnlyBuffer | WriteableBuffer  # stable
+ ReadOnlyBuffer: TypeAlias = Buffer  # stable
+ ReadableBuffer: TypeAlias = Buffer  # stable

Before: https://github.com/python/typeshed/blame/56aeeb677f27c6771e683314621521aff5c97cd1/stdlib/_typeshed/__init__.pyi#L230-L241
After: https://github.com/python/typeshed/blame/1e7e174b4fc4da6a66622168f8012c7ee0499392/stdlib/_typeshed/__init__.pyi#L263-L267


Deferred until #11422.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the urllib.request.Request data annotation and the Buffer aliases in stdlib/_typeshed/init.pyi, comparing the before and after changes linked in the issue. Reproduce the supplied mypy example and review PR #10225 and deferred issue #11422 for context. Done means the example accepts decode without losing the intended accepted data types.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.