chakra-core / chakra-core/ChakraCore

Consider UTF-8?

Open
#6,570 12 comments 0 reactions 0 assignees View on GitHub
Discussion
Dominant language
JavaScript
Stars
9.3k
Forks
1.2k
PR merge metrics
No merged PRs in 30d

Description

We are using UTF-16, which has some cross platform compatibility issues, starting with different type used on different platforms - `wchar_t` on Windows and `char16_t` everywhere else, with former being a fundamental type (not requiring an include), but latter - defined as an unsigned short. This is thanks to 'nix toolchains picking UTF-32 (which is excessive for most uses) as default Unicode type.

UTF-8 is more cross platform, and supported in Windows [since Win10][w]. For prior versions of Windows it should be possible to convert to UTF-16 for things that require it. A quick search shows that V8 might use UTF-8.

I am not sure whether or not UTF-8 would be easier for our text-related projects, like RegEx support. @rhuanjl do you have any thoughts?

[w]: https://en.wikipedia.org/wiki/Unicode_in_Microsoft_Windows#UTF-8

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.