facebook / facebook/rocksdb

bottommost_compression doesn't reject unsupported compression types

Open
#3,147 0 comments 0 reactions 0 assignees View on GitHub
bug up-for-grabs
Dominant language
C++
Stars
32.1k
Forks
6.9k
Avg merge
32m
Merged PRs (30d)
1

Description

I think this explains a user problem. They set bottommost_compression to kZSTD, but that is not supported, so they don't get compression and use 10X more space than expected.
https://groups.google.com/forum/#!topic/myrocks-dev/N95FCR2UGoA

My server supports kZSTDNotFinalCompression but not kZSTD.

When I try to set compression for a level to kZSTD the server doesn't start as expected:
```
[ERROR] RocksDB: Error opening instance, Status Code: 4, Status: Invalid argument: Compression type ZSTD is not linked wit
h the binary.
```
When I set bottommost_compression to kZSTD the server starts.

This is rejected, as expected:
```
rocksdb_default_cf_options=compression_per_level=kLZ4Compression:kZSTD;bottommost_compression=kLZ4Compression
```

This not rejected, as expected, because the compression types are supported:
```
rocksdb_default_cf_options=compression_per_level=kLZ4Compression:kLZ4Compression;bottommost_compression=kZSTDNotFinalCompression
```

This is accepted but should be rejected because kZSTD is not supported:
```
rocksdb_default_cf_options=compression_per_level=kLZ4Compression:kLZ4Compression;bottommost_compression=kZSTD
```

From RocksDB LOG
```
2017/11/08-10:42:07.701440 7f7a7fc8da40 Compression algorithms supported:
2017/11/08-10:42:07.701441 7f7a7fc8da40 Snappy supported: 1
2017/11/08-10:42:07.701442 7f7a7fc8da40 Zlib supported: 1
2017/11/08-10:42:07.701443 7f7a7fc8da40 Bzip supported: 0
2017/11/08-10:42:07.701444 7f7a7fc8da40 LZ4 supported: 1
2017/11/08-10:42:07.701445 7f7a7fc8da40 ZSTDNotFinal supported: 1
2017/11/08-10:42:07.701451 7f7a7fc8da40 ZSTD supported: 0
...
2017/11/08-10:42:07.701662 7f7a7fc8da40 Options.compression[0]: LZ4
2017/11/08-10:42:07.701664 7f7a7fc8da40 Options.compression[1]: LZ4
2017/11/08-10:42:07.701665 7f7a7fc8da40 Options.bottommost_compression: ZSTD
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.