python / python/cpython

Update FileIO comments, documentation to match implementation

Open
#129,011 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

docs
Dominant language
Python
Stars
77.2k
Forks
35.9k
PR merge metrics
PR metrics pending

Description

Documentation

Discuss thread: https://discuss.python.org/t/change-open-write-to-guarantee-all-bytes-will-be-written-or-an-exception-will-be-thrown/71082

  1. FileIO behaves differently than its code comments around read and readall. Update the code comments to match current behavior
    • PEP 475, reads are retried in some cases
    • .readall() makes multiple system calls by design
  2. Add documentation to open() builtin that buffering=0, which currently just says "disables buffering") changes how .write() behaves and may result in data loss as a result of a partial write (ex. gh-126606).
    • TextIO and BufferedIO (which are gotten via commonly used open('README.rst'), open('README.rst', 'rb')) retry partial writes providing a user a guarantee either all bytes will be written or an exception will be thrown.
    • FileIO is "Raw I/O" and by PEP-3116 design, as documented, and currently implemented does not retry partial writes.
    • Most the time, buffering=0 currently speeds up writing a file, but it can also result in corrupted files, ex. gh-126606 from using FileIO directly
    • I would like to try and change the behavior of buffering=0 to use BufferedIO but with a 0 sized buffer, and that is on my roadmap but will be a while (people use the flag for a reason / get benefit!). This will mean open() always returns an object which implements "Write all or throw exception" behavior.
    • Add warning / document existing behavior on open() in the meantime.
Linked PRs
  • gh-129012
  • gh-135328
  • gh-150957
  • gh-150958
  • gh-150959

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the FileIO code comments and the open() builtin documentation linked in the issue, then inspect linked PRs gh-129012, gh-135328, gh-150957, gh-150958, and gh-150959. Done means the comments describe current read/readall behavior and the open() documentation clearly explains buffering=0 and partial-write data-loss behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
documentation
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.