python / python/cpython

Confusing behavior of selectors with buffered file objects

Open
#101,053 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

docs topic-socket
Dominant language
Python
Stars
77.2k
Forks
36k
PR merge metrics
PR metrics pending

Description

Using selectors with buffered file objects is confusing because the selector will only report whether there is something to read in the underlying file descriptor and ignore the internal buffer. This is especially relevant because buffered file objects are used in many places in the standard library, e.g. sys.stdin or socket.makefile().

I propose to add a corresponding warning to the docs, e.g. like this:

Selectors only look at the underlying file descriptors, so they do not know about data that is buffered in python code. Buffered file objects are used in many places in the standard library, e.g. sys.stdin or socket.makefile(). In those cases, it is recommended to bypass the buffering by using os.read(fileobj.fileno(), size) instead of fileobj.read(size).

Even better would be add code to handle buffered file objects. Unfortunately, I think that BufferedIOBase does not expose whether it has data in its buffer.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the selectors documentation and the BufferedIOBase behavior referenced in the issue, then compare examples using sys.stdin and socket.makefile(). Determine whether the scope is a documentation warning or implementation support for buffered data. Done means the chosen behavior is documented clearly, including the os.read(fileobj.fileno(), size) recommendation if the change remains documentation-only.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
documentation
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.