PaperMC / PaperMC/Paper

Mismatched encoding between console and texts got printed results in garbled texts

Open
#13,865 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

status: needs triage
Dominant language
Java
Stars
12.7k
Forks
3.5k
Avg merge
3d 13h
Merged PRs (30d)
11

Description

Is your feature request related to a problem?

This issue was suggested by @jpenilla at https://github.com/jpenilla/run-task/pull/61#issuecomment-4340967269
Tried to discuss the details I should know before writing the issue description but got no response, so I'm here first. Ref: https://discord.com/channels/289587909051416579/555462289851940864/1500492644906700952
Don't know if it is a feature or bug. So sorry if I placed this under incorrect label.

In short: Java runtime reads the default locale from system, but modern Java code always use UTF-8 for strings even when printing, results in garbled texts (actually wrongly-encoded texts) printed in console if the original text is non-ASCII and not encodable with system charset.

As far as I know, the issue was long-lasting in Chinese community because the Microsoft(R) Windows builds for Simplified Chinese uses GBK/GB2312 as its default charset, which affects Java runtime unless file.encoding property is used to force JRE use different charset for file I/O. sout/serr is device file so the property applys to their I/O as well.

Describe the solution you'd like.

We might need some native level tricks to change the codepage the terminal uses, which is what I don't know much about. Another solution might be re-opening sout/serr at main method by creating new file output stream objects using specified charset.

Describe alternatives you've considered.

Let JRE use different charset for default case -- that is already what I had done, but @jpenilla thinks it affects wider than this issue itself, so I'm here to discuss another solution.
Ask users to change their terminal encoding -- I think there is a way to be done programmatically without user operation, so a fix for this issue will benefit webpanel users or new server owners.

Other

The reason I decide to do fix work on run-task is because of the range this issue affects -- it does NOT only apply to Minecraft server software, but may even ALL applications.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing non-ASCII output on a Windows system using a GBK/GB2312 default charset, then trace how sout and serr are created or handled at the main method. Compare terminal code-page changes with reopening the streams using an explicit charset. Done should mean correctly rendered output without requiring users to change file.encoding or terminal settings.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
cli
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.