google / google/leveldb

Unicode filename handling in new windows environment

Open
#755 8 comments 5 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
39.4k
Forks
8.2k
PR merge metrics
No merged PRs in 30d

Description

The new windows environment uses the *A functions instead of the *W ones:
```c++
size_t error_text_size = ::FormatMessageA(
ScopedHandle handle = ::CreateFileA(
::CreateFileA(filename.c_str(), desired_access, share_mode,
::CreateFileMappingA(handle.get(),
ScopedHandle handle = ::CreateFileA(
ScopedHandle handle = ::CreateFileA(
return GetFileAttributesA(filename.c_str()) != INVALID_FILE_ATTRIBUTES;
HANDLE dir_handle = ::FindFirstFileA(find_pattern.c_str(), &find_data);
} while (::FindNextFileA(dir_handle, &find_data));
if (!::DeleteFileA(filename.c_str())) {
if (!::CreateDirectoryA(dirname.c_str(), nullptr)) {
if (!::RemoveDirectoryA(dirname.c_str())) {
if (!::GetFileAttributesExA(filename.c_str(), GetFileExInfoStandard,
if (::MoveFileA(from.c_str(), to.c_str())) {
if (::ReplaceFileA(to.c_str(), from.c_str(), /*lpBackupFileName=*/nullptr,
ScopedHandle handle = ::CreateFileA(
if (!GetTempPathA(ARRAYSIZE(tmp_path), tmp_path)) {
```
I think these use what the process code page happens to be, which might not be UTF-8. I don't claim to understand Windows unicode handling, so please tell me I'm wrong, but in our [previous Windows environment](https://github.com/bitcoin-core/leveldb/blob/bitcoin-fork/util/env_win.cc) we used

```c++
void ToWidePath(const std::string& value, std::wstring& target) {
wchar_t buffer[MAX_PATH];
MultiByteToWideChar(CP_UTF8, 0, value.c_str(), -1, buffer, MAX_PATH);
target = buffer;
}

void ToNarrowPath(const std::wstring& value, std::string& target) {
char buffer[MAX_PATH];
WideCharToMultiByte(CP_UTF8, 0, value.c_str(), -1, buffer, MAX_PATH, NULL, NULL);
target = buffer;
}
```
along with the *W wide variants of Windows API functions (but https://github.com/google/leveldb/issues/755#issuecomment-559721707 is a better way using std::codecvt).

I'm seeing some downstream test failures regarding file names, and I suspect this might be the cause.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.