python / python/cpython

`mailbox.MH.__setitem__()` can destroy a message when replacement fails

Open
#156,312 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

stdlib topic-email type-bug
Dominant language
Python
Stars
77.2k
Forks
35.9k
PR merge metrics
PR metrics pending

Description

Bug description:
Summary

`When mailbox.MH replaces an existing message with an invalid str, it correctly raises ValueError, but the original message file may already have been truncated to empty. In other words, the replacement fails while also destroying the existing message content, resulting in data loss.

Affected public API: mailbox.MH.__setitem__().

Minimal reproducer

Run this against CPython before the fix:

import mailbox
import tempfile

with tempfile.TemporaryDirectory() as path:
    box = mailbox.MH(path)
    key = box.add(b"Subject: original\n\noriginal body\n")
    original = box.get_bytes(key)

    try:
        box[key] = "Subject: caf\u00e9\n\nreplacement body\n"
    except ValueError as exc:
        print(type(exc).__name__, exc)

    print("in-memory:", box.get_bytes(key))
    box.close()

    reopened = mailbox.MH(path)
    print("reopened:", reopened.get_bytes(key))
    assert reopened.get_bytes(key) == original

Actual result before the fix:

ValueError String input must be ASCII-only; use bytes or a Message instead
in-memory: b''
reopened: b''
AssertionError

Expected result:

ValueError String input must be ASCII-only; use bytes or a Message instead
in-memory: b'Subject: original\n\noriginal body\n'
reopened: b'Subject: original\n\noriginal body\n'

The same problem occurs if a file-like message object raises while it is being read: the old message is replaced with the bytes written before the exception.

Root cause

MH.__setitem__() performed these operations in this order:

open existing message
-> open the same path with O_TRUNC
-> serialize the replacement with _dump_message()
-> propagate a serialization error

O_TRUNC changes the existing message file before _dump_message() validates or fully reads the replacement. For a non-ASCII str, _dump_message() calls _string_to_bytes() and raises ValueError immediately, leaving the already truncated file in place. There is no rollback path.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs
  • gh-156313

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with mailbox.MH.setitem() and _dump_message(), then run the minimal reproducer from the issue on CPython's mailbox tests. Done means failed replacement of a non-ASCII string or raising file-like message leaves both the in-memory and reopened original message unchanged.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.