facebook / facebook/rocksdb

DB name ending with a non-ASCII character causes assertion

Open
#7,809 1 comment 1 reaction 0 assignees View on GitHub
bug up-for-grabs
Dominant language
C++
Stars
32.1k
Forks
6.9k
Avg merge
32m
Merged PRs (30d)
1

Description

### Expected behavior

After providing a DB name ending with a non-ASCII character, the DB should open without any errors or assertions.

### Actual behavior

After providing a DB name ending with a non-ASCII character, one gets an assertion on Windows -- even if WITH_WINDOWS_UTF8_FILENAMES is set to ON. Tested on Microsoft Windows (Version 10.0.18363.1256) with Visual Studio 2019 Developer Command Prompt v16.6.1 and RocksDB version 6.14.6. (Please also note that if you have a DB name containing non-trailing non-ASCII characters then everything is fine.)

![image](https://user-images.githubusercontent.com/875677/103012157-6f711100-453b-11eb-8847-f56f96d346c3.png)

### Steps to reproduce the behavior

Please refer to the following minimal stand-alone example repoducing the issue:

* [https://gist.github.com/katusk/79a1b5937e5f404a13053f5431159884](https://gist.github.com/katusk/79a1b5937e5f404a13053f5431159884)

On Windows, you can run `test.bat` from a "Developer Command Prompt for VS 2019" assuming you have Visual Studio 2019 with CMake installed.

You can reproduce the assertion by simply calling `rocksdb_open(pRocksDBOptions, u8"Bú", &rocksDBError);` with a DB name containing a trailing non-ASCII character. If the trailing character is an ASCII character, then we are fine -- so u8"Búb" or u8"Bu" are fine -- you will not get an assertion. The culprit seems to be the `RocksDBOptionsParser::TrimAndRemoveComment` function called when the temporary options file is parsed and verified: even though the options file is properly encoded in UTF8, and the `line` parameter of the function contains a valid UTF8 string, the following code gives an assertion for a `line` parameter with the value of ` wal_dir=Bú`:

```
// start < end implies end > 0.
while (start < end && isspace(line[end - 1]) != 0) {
--end;
}
```

This is because the `isspace` function seems to expect an ASCII character. As a quick hack, you can circumvent this expectation by simply:

```
// start < end implies end > 0.
while (start < end && line[end - 1] >= -1 && line[end - 1] <= 255 && isspace(line[end - 1]) != 0) {
--end;
}
```

And with latter modifications a DB with u8"Bú" opens just fine. There should be a better solution than this though.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.