godotengine / godotengine/godot-cpp

String::String(const char*) is expecting latin1 instead of UTF-8

Open
#784 7 comments 2 reactions 0 assignees View on GitHub
bug topic:gdextension
Dominant language
C++
Stars
2.7k
Forks
809
Avg merge
1d 3h
Merged PRs (30d)
8

Description

Since #695, `String::String(const char*)` is now internally calling `string_new_with_latin1_chars` instead of `string_new_with_utf8_chars`.

diff: https://github.com/godotengine/godot-cpp/commit/bf8fc4c53d07cf8ac4d8343364899c197076fbc2#diff-20a0dfd8072691f654c12adb6aecabee9d3247fb99488b37539dab1d967c97b8L77-L79

Because of this change, we can't pass non-latin1 characters for String-like parameters anymore.
For example, `node->get_node("Path/To/NonLatin1/名前")` fails.

Technically speaking, `char*` is encoded in a native encoding, which is not guaranteed to be neither UTF-8 nor Latin-1 (by definition). Thus, we should not choose between `latin1` and `utf8` in the first place; we should add a new function `string_new_with_native_chars` and call it whenever `char*` is used as an input. But that's a very long story. For sake of simplicity, we can just expect UTF-8 for `char*`, because it is the default for many environments today (note that MSVC can be configured to use UTF-8 by the `/utf-8` compiler flag).

No offense, but even if the change of the default encoding was intended, it should not be done in #695 because it is a breaking change. I consider this a regression.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.