aboutcode-org / aboutcode-org/scancode-toolkit

compiledcode: java class strings are not utf-8 but mutf-8

Open
#3,145 3 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
2.6k
Forks
791
Avg merge
1d 12h
Merged PRs (30d)
5

Description

### Description

The parser for Java byte code is incorrect: it treats strings (incl method names, class names, and so on) as UTF-8, but according to the Java class specification these are MUTF-8:

While for *most* strings they are interchangeable it will be different once you start getting non-Western strings (example: CJK). If strings are important to you, then you should add another step and that's translating the strings. For this I am using the mutf8 package although I have a feeling that there are still a few bugs in that package (that I need to chase).

MUTF-8 is also used in Android byte code.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.