email with multipart/related content does not round-trip parsebytes/as_bytes
Open
Nobody has claimed this yet.
stdlib
topic-email
type-bug
- Dominant language
- Python
- Stars
- 77.2k
- Forks
- 35.9k
- PR merge metrics
- PR metrics pending
Description
Bug report
Bug description:
import email
import email.parser
import email.policy
with open('sample_email.eml', 'rb') as f:
raw_email = f.read()
policy = email.policy.default.clone(cte_type='8bit', utf8=True, refold_source='none')
parser = email.parser.BytesParser(policy=policy)
decoded_email = parser.parsebytes(raw_email)
print("Defects: %s" % decoded_email.defects)
reencoded_email = decoded_email.as_bytes(policy=policy)
print("Two messages matches: %s" % (raw_email == reencoded_email))
sample_email.txt (save it as sample_email.eml)
When ran on different python versions (including 3.13), it raises UnicodeEncodeError:
Defects: []
Traceback (most recent call last):
File "/var/tmp/etienne/sample_email.py", line 13, in <module>
reencoded_email = decoded_email.as_bytes(policy=policy)
File "/usr/lib/python3.13/email/message.py", line 208, in as_bytes
g.flatten(self, unixfrom=unixfrom)
~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.13/email/generator.py", line 117, in flatten
self._write(msg)
~~~~~~~~~~~^^^^^
File "/usr/lib/python3.13/email/generator.py", line 182, in _write
self._dispatch(msg)
~~~~~~~~~~~~~~^^^^^
File "/usr/lib/python3.13/email/generator.py", line 219, in _dispatch
meth(msg)
~~~~^^^^^
File "/usr/lib/python3.13/email/generator.py", line 286, in _handle_multipart
g.flatten(part, unixfrom=False, linesep=self._NL)
~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.13/email/generator.py", line 117, in flatten
self._write(msg)
~~~~~~~~~~~^^^^^
File "/usr/lib/python3.13/email/generator.py", line 182, in _write
self._dispatch(msg)
~~~~~~~~~~~~~~^^^^^
File "/usr/lib/python3.13/email/generator.py", line 219, in _dispatch
meth(msg)
~~~~^^^^^
File "/usr/lib/python3.13/email/generator.py", line 278, in _handle_multipart
self.write(subparts)
~~~~~~~~~~^^^^^^^^^^
File "/usr/lib/python3.13/email/generator.py", line 420, in write
self._fp.write(s.encode('ascii', 'surrogateescape'))
~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeEncodeError: 'ascii' codec can't encode characters in position 330-333: ordinal not in range(128)
CPython versions tested on:
3.13
Operating systems tested on:
Linux
Linked PRs
- gh-136848
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the example with email.parser.BytesParser and the supplied sample_email.eml, then trace email/message.py into email/generator.py, especially _handle_multipart and write. Done means as_bytes(policy=policy) completes without UnicodeEncodeError and the reencoded bytes match the original input; review linked PR gh-136848 before starting.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100