Switch to UTF-8
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 3.6k
- Forks
- 536
- Avg merge
- 6d 1h
- Merged PRs (30d)
- 3
Description
Rationale
Currently, engine uses system native locales and just doesn't support UTF-8. This results in several problems, like player save files having wrong characters when written on disk, inconvenient interoperability with third party libraries that use UTF-8 by default.
Proposed solution
To my knowledge, C++17/C++20 doesn't provide required UTF-8 functionality.
There are two big libraries that can be used:
Their functionality should be equal, ICU is preferred, and we likely want to support both (i.e. iconv too), because e.g. iconv is native to some systems and doesn't require additional packages to be installed.
Useful links
FreeBSD documentation:
- https://docs.freebsd.org/en/books/porters-handbook/special/#using-iconv
- https://people.freebsd.org/~olivierd/porters-handbook/using-iconv.html
- https://docs-archive.freebsd.org/doc/9.3-RELEASE/usr/share/doc/freebsd/ru/books/porters-handbook/using-iconv.html
Methodology
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No files, tests, or entry points are identified in the issue. First map where the engine handles system locales, save-file I/O, and third-party text boundaries, then compare ICU and iconv support across platforms; done means consistent UTF-8 handling and correct save-file interoperability.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- game-dev, internationalization
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100