dotnet / dotnet/fsharp

Compiler does not handle non-BMP characters in identifiers

Open
#9,600 7 comments 1 reaction 0 assignees View on GitHub
Area-Compiler-Syntax Feature Request
Dominant language
F#
Stars
4.3k
Forks
876
Avg merge
4d 11h
Merged PRs (30d)
131

Description

The compiler uses methods on ``Char`` extensively to check character properties, even in recent code such as #9199. This breaks down on non-BMP characters, and these properties should be checked on the full 32-bit code points instead. A similar bug exists in Roslyn, but it has been acknowledged as a bug (given that the C# standard allows non-BMP characters identifiers), and is in the process of being [fixed](https://github.com/ufcpp/roslyn/commit/59df36d896bf392c1a13243b5141d9b1460cbd50). Therefore, for interoperability with C#, even putting aside the many minority languages around the world whose scripts are outside the BMP and the many CJK personal and place names outside of the BMP, these identifiers will need to be handled properly.

# Example of the issue
```fsharp
let δ𐌰 = 5
```
Although the Greek letter δ is handled properly, the Gothic letter 𐌰 causes the compiler to explain of an unexpected character in the pattern.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.