oracle / oracle/graalpython

[Bug]: Different output upon decoding invalid utf-8

Open
#875 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
1.6k
Forks
155
Avg merge
9h 42m
Merged PRs (30d)
36

Description

Describe the bug

Not sure whether it is a bug or just a different behaviour.
Not critical either way, but thought filing it for reference anyway.

  • When decoding invalid UTF-8 with errors='replace', CPython (and PyPy for that matter) decodes every invalid byte as .
  • GraalPy, OTOH, treats invalid UTF-8 triplets as one character.
Operating system

Linux

CPU architecture

x86_64

GraalPy version

GraalPy 3.12.8 (GraalVM CE Native 25.0.2)

JDK version

No response

Context configuration

No response

Steps to reproduce

GraalPy:

$ graalpy -c "print(b'\xed\xae\x80\xed\xb0\x80'.decode(errors='replace'))"
��

CPython:

$ python -c "print(b'\xed\xae\x80\xed\xb0\x80'.decode(errors='replace'))"
������
Expected behavior

To match CPython unless there is a good reason not to.

Stack trace

Additional context

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the reported graalpy -c decoding reproducer and compare its errors='replace' output with CPython for the supplied invalid UTF-8 bytes. Trace the decode entry point to identify where invalid triplets are grouped, then verify that the output matches CPython's replacement behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.