objectionary / objectionary/lints

`ascii-only` defect message misrepresents supplementary Unicode characters

Open
#1,446 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug good-title
Dominant language
Java
Stars
14
Forks
39
Avg merge
22h 54m
Merged PRs (30d)
90

Description

What happens

LtAsciiOnly identifies the first forbidden comment character with String.chars(), then narrows the returned UTF-16 code unit to a single char:

final Optional<Character> abusive = comment.text().get().chars()
    .filter(chr -> (chr < 32 && chr != '\n') || chr > 127)
    .mapToObj(chr -> (char) chr)
    .findFirst();
...
final Character chr = abusive.get();
...
String.format(
    "Only ASCII characters are allowed in comments, while \"%s\" is used ...",
    chr,
    ...
)

On current master 91ff81b80f0918d1e01846f289ead22dc58ad3e0, a supplementary Unicode character such as 😀 is represented by a surrogate pair. The first surrogate is selected as the “character”, detached from its pair, and the diagnostic no longer contains the offending character.

A minimal reproduction of the exact current algorithm:

String text = "hello 😀 world";
Character bad = text.chars()
    .filter(chr -> (chr < 32 && chr != '\n') || chr > 127)
    .mapToObj(chr -> (char) chr)
    .findFirst()
    .orElseThrow();
System.out.println(Integer.toHexString(bad));
System.out.println(bad);
System.out.println(Integer.toHexString(text.codePointAt(text.indexOf("😀"))));

Actual output on Java 21:

bad-code-unit=d83d
bad-rendered=?
actual-codepoint=1f600

So a comment containing 😀 is reported as containing ?/an unpaired surrogate rather than U+1F600.

Why this is wrong

The lint message explicitly tells the user which character is forbidden. For BMP characters (Cyrillic/Chinese), the existing tests pass because one UTF-16 code unit equals one Unicode code point. Supplementary characters are different: String.chars() enumerates code units, not code points, and narrowing one surrogate to char destroys the original character.

This also makes the implementation inconsistent with its semantic task: the rule is about characters/code points, not Java UTF-16 storage units.

What should happen

Use String.codePoints() and retain the offending value as an int code point. Render it with Character.toChars(codePoint) (or an equivalent code-point-safe conversion), and compute the diagnostic position from the code point's UTF-16 offset without splitting the surrogate pair.

A regression fixture with an emoji in the top comment should assert that the defect message contains the actual emoji rather than a replacement/unpaired-surrogate glyph.

I searched existing issues for ascii-only emoji, ascii-only surrogate, LtAsciiOnly supplementary, and ASCII comment emoji position and found no duplicate.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the LtAsciiOnly implementation and its existing regression fixtures for comment diagnostics. Run the relevant linter tests, then verify that an emoji in the top comment is reported as the actual character and that its diagnostic position does not split the surrogate pair.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.