objectionary / objectionary/lints
`ascii-only` defect message misrepresents supplementary Unicode characters
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 14
- Forks
- 39
- Avg merge
- 22h 54m
- Merged PRs (30d)
- 90
Description
What happens
LtAsciiOnly identifies the first forbidden comment character with String.chars(), then narrows the returned UTF-16 code unit to a single char:
final Optional<Character> abusive = comment.text().get().chars()
.filter(chr -> (chr < 32 && chr != '\n') || chr > 127)
.mapToObj(chr -> (char) chr)
.findFirst();
...
final Character chr = abusive.get();
...
String.format(
"Only ASCII characters are allowed in comments, while \"%s\" is used ...",
chr,
...
)
On current master 91ff81b80f0918d1e01846f289ead22dc58ad3e0, a supplementary Unicode character such as 😀 is represented by a surrogate pair. The first surrogate is selected as the “character”, detached from its pair, and the diagnostic no longer contains the offending character.
A minimal reproduction of the exact current algorithm:
String text = "hello 😀 world";
Character bad = text.chars()
.filter(chr -> (chr < 32 && chr != '\n') || chr > 127)
.mapToObj(chr -> (char) chr)
.findFirst()
.orElseThrow();
System.out.println(Integer.toHexString(bad));
System.out.println(bad);
System.out.println(Integer.toHexString(text.codePointAt(text.indexOf("😀"))));
Actual output on Java 21:
bad-code-unit=d83d
bad-rendered=?
actual-codepoint=1f600
So a comment containing 😀 is reported as containing ?/an unpaired surrogate rather than U+1F600.
Why this is wrong
The lint message explicitly tells the user which character is forbidden. For BMP characters (Cyrillic/Chinese), the existing tests pass because one UTF-16 code unit equals one Unicode code point. Supplementary characters are different: String.chars() enumerates code units, not code points, and narrowing one surrogate to char destroys the original character.
This also makes the implementation inconsistent with its semantic task: the rule is about characters/code points, not Java UTF-16 storage units.
What should happen
Use String.codePoints() and retain the offending value as an int code point. Render it with Character.toChars(codePoint) (or an equivalent code-point-safe conversion), and compute the diagnostic position from the code point's UTF-16 offset without splitting the surrogate pair.
A regression fixture with an emoji in the top comment should assert that the defect message contains the actual emoji rather than a replacement/unpaired-surrogate glyph.
I searched existing issues for ascii-only emoji, ascii-only surrogate, LtAsciiOnly supplementary, and ASCII comment emoji position and found no duplicate.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the LtAsciiOnly implementation and its existing regression fixtures for comment diagnostics. Run the relevant linter tests, then verify that an emoji in the top comment is reported as the actual character and that its diagnostic position does not split the surrogate pair.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100