microsoft / microsoft/monaco-editor

Rust: Hexadecimal literals with underscores after hex letters not highlighted correctly

Open
#4,917 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
46.8k
Forks
4.1k
Avg merge
17h 58m
Merged PRs (30d)
1

Description

Issue Description

Rust hexadecimal literals containing underscores after hex letters (a-f) lose syntax highlighting after the underscore. For example, in 0x01_02_0a_0b, the highlighting stops working correctly after the underscore that follows the hex letter a, so a_0b loses its number highlighting.

Expected Behavior

The entire hexadecimal literal 0x01_02_0a_0b should be highlighted as a number throughout.

Current Behavior

Only 0x01_02_0 is highlighted as a number, while a_0b loses the number highlighting.

Root Cause

In the Rust language definition at src/basic-languages/rust/rust.ts, line 344, the hexadecimal pattern is:

[/(0x[\da-fA-F]+)_?(@intSuffixes)?/, { token: "number" }],

The problem is with _? - it only allows one underscore at the end. However, the pattern [\da-fA-F]+ stops matching after encountering an underscore that follows a hex letter, causing the tokenizer to break the number highlighting.

Proposed Fix

Change the pattern to allow underscores throughout the hex number, similar to how octal and binary literals are handled:

[/(0x[0-9a-fA-F_]+)(@intSuffixes)?/, { token: "number" }],

This matches the patterns used for other number formats in the same file:

  • Octal: [/(0o[0-7_]+)(@intSuffixes)?/, { token: "number" }], (line 336)
  • Binary: [/(0b[0-1_]+)(@intSuffixes)?/, { token: "number" }], (line 338)

Test Cases

The following Rust code should have proper syntax highlighting:

let hex1 = 0x01_02_0a_0b;     // Currently broken
let hex2 = 0xFF_AA_BB_CC;     // Currently broken  
let hex3 = 0x1234_abcd_5678;  // Currently broken
let hex4 = 0x123456;          // Currently works
let hex5 = 0x123456_;         // Currently works

Reference

This issue was originally reported in the Compiler Explorer project: https://github.com/compiler-explorer/compiler-explorer/issues/7645

Rust Language Specification

According to the Rust Reference, underscores can be used as visual separators anywhere within integer literals, including hexadecimal ones.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Open src/basic-languages/rust/rust.ts around line 344 and compare the hexadecimal rule with the octal and binary rules at lines 336 and 338. Confirm the change by checking that the listed hexadecimal literals, including underscores after hex letters, remain highlighted as numbers throughout.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, typescript
Domain
tooling
Issue type
Bug
Difficulty
1/5
Estimated time
Under an hour
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.