microsoft / microsoft/monaco-editor
Rust: Hexadecimal literals with underscores after hex letters not highlighted correctly
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 46.8k
- Forks
- 4.1k
- Avg merge
- 17h 58m
- Merged PRs (30d)
- 1
Description
Issue Description
Rust hexadecimal literals containing underscores after hex letters (a-f) lose syntax highlighting after the underscore. For example, in 0x01_02_0a_0b, the highlighting stops working correctly after the underscore that follows the hex letter a, so a_0b loses its number highlighting.
Expected Behavior
The entire hexadecimal literal 0x01_02_0a_0b should be highlighted as a number throughout.
Current Behavior
Only 0x01_02_0 is highlighted as a number, while a_0b loses the number highlighting.
Root Cause
In the Rust language definition at src/basic-languages/rust/rust.ts, line 344, the hexadecimal pattern is:
[/(0x[\da-fA-F]+)_?(@intSuffixes)?/, { token: "number" }],
The problem is with _? - it only allows one underscore at the end. However, the pattern [\da-fA-F]+ stops matching after encountering an underscore that follows a hex letter, causing the tokenizer to break the number highlighting.
Proposed Fix
Change the pattern to allow underscores throughout the hex number, similar to how octal and binary literals are handled:
[/(0x[0-9a-fA-F_]+)(@intSuffixes)?/, { token: "number" }],
This matches the patterns used for other number formats in the same file:
- Octal:
[/(0o[0-7_]+)(@intSuffixes)?/, { token: "number" }],(line 336) - Binary:
[/(0b[0-1_]+)(@intSuffixes)?/, { token: "number" }],(line 338)
Test Cases
The following Rust code should have proper syntax highlighting:
let hex1 = 0x01_02_0a_0b; // Currently broken
let hex2 = 0xFF_AA_BB_CC; // Currently broken
let hex3 = 0x1234_abcd_5678; // Currently broken
let hex4 = 0x123456; // Currently works
let hex5 = 0x123456_; // Currently works
Reference
This issue was originally reported in the Compiler Explorer project: https://github.com/compiler-explorer/compiler-explorer/issues/7645
Rust Language Specification
According to the Rust Reference, underscores can be used as visual separators anywhere within integer literals, including hexadecimal ones.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Open src/basic-languages/rust/rust.ts around line 344 and compare the hexadecimal rule with the octal and binary rules at lines 336 and 338. Confirm the change by checking that the listed hexadecimal literals, including underscores after hex letters, remain highlighted as numbers throughout.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust, typescript
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 1/5
- Estimated time
- Under an hour
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 58/100