python / python/cpython

ValueError: illegal newline value on 0x85 (NEL)

Open
#97,544 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

interpreter-core type-feature
Dominant language
Python
Stars
77.2k
Forks
36k
PR merge metrics
PR metrics pending

Description

In attempting to harmonize the path pie implementation with more modern constructs, I'm seeking also to utilize the newline argument to open instead of explicitly substituting characters.

When I do, however, I find that some tests start to fail when NEL is passed as a newline character:

$ py -c "open('foo', 'w', newline='\x85')"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ValueError: illegal newline value: …

At least in some contexts, Python recognizes NEL as a line separator:

>>> 'a\x85b\x1ec'.splitlines()
['a', 'b', 'c']

See also #66428 for some background.

According to the wikipedia article, a conforming application should honor a number of characters as newlines:

>>> newlines = '\x0a', '\x0b', '\x0c', '\x0d', '\x0d\x0a', '\u0085', '\u2028', '\u2029'
>>> for candidate in newlines:
...   try: x=open('foo', 'w', newline=candidate)
...   except ValueError: print('failed on', repr(candidate))
... 
failed on '\x0b'
failed on '\x0c'
failed on '\x85'
failed on '\u2028'
failed on '\u2029'

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the reported open(..., newline=...) reproducer and compare it with str.splitlines() for the listed newline characters. Trace the CPython text I/O handling for newline validation, then identify or add the relevant tests. Done means the intended newline values have defined, tested behavior without regressing existing newline translation.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.