[Bug]: Different output upon decoding invalid utf-8
Open
Nobody has claimed this yet.
bug
- Dominant language
- Python
- Stars
- 1.6k
- Forks
- 155
- Avg merge
- 9h 42m
- Merged PRs (30d)
- 36
Description
Describe the bug
Not sure whether it is a bug or just a different behaviour.
Not critical either way, but thought filing it for reference anyway.
- When decoding invalid UTF-8 with
errors='replace', CPython (and PyPy for that matter) decodes every invalid byte as�. - GraalPy, OTOH, treats invalid UTF-8 triplets as one character.
Operating system
Linux
CPU architecture
x86_64
GraalPy version
GraalPy 3.12.8 (GraalVM CE Native 25.0.2)
JDK version
No response
Context configuration
No response
Steps to reproduce
GraalPy:
$ graalpy -c "print(b'\xed\xae\x80\xed\xb0\x80'.decode(errors='replace'))"
��
CPython:
$ python -c "print(b'\xed\xae\x80\xed\xb0\x80'.decode(errors='replace'))"
������
Expected behavior
To match CPython unless there is a good reason not to.
Stack trace
Additional context
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the reported graalpy -c decoding reproducer and compare its errors='replace' output with CPython for the supplied invalid UTF-8 bytes. Trace the decode entry point to identify where invalid triplets are grouped, then verify that the output matches CPython's replacement behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100