facebook / facebook/rocksdb

Compaction stats length is limited to 65536

Open
#9,353 2 comments 1 reaction 1 assignee Claimed by @lhsoft View on GitHub
feature-request
Dominant language
C++
Stars
32.1k
Forks
6.9k
Avg merge
32m
Merged PRs (30d)
1

Description

### Expected behavior

Get full compaction logs after calling `DBImpl::DumpStats`: https://github.com/facebook/rocksdb/blob/9ef7f70c11e26ac82bc3716d47902ad3a773fa48/db/db_impl/db_impl.cc#L893

### Actual behavior

Instead, compaction stats length is limited to 65536, which results in only partial logs display (in reality we have > 50 columns):

```
** DB Stats **
Uptime(secs): 24069.1 total, 600.0 interval
Cumulative writes: 3279K writes, 17M keys, 3279K commit groups, 1.0 writes per commit group, ingest: 13.45 GB, 0.57 MB/s

...

** Compaction Stats [col22] **
Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
L0 1/0 16.68 MB 0.5 0.0 0.0 0.0 0.0
```

### Steps to reproduce the behavior

Call `DBImpl::DumpStats` with large number of columns (200 should be enough).

### Idea

This line is causing cropping compaction stats: https://github.com/facebook/rocksdb/blob/2e09a54c4fb82e88bcaa3e7cfa8ccbbbbf3635d5/logging/posix_logger.h#L100

The simplest solution could be increasing the buffer, but it could be more robust to print compaction stats separately for each column instead of collecting all stats to one string: https://github.com/facebook/rocksdb/blob/9ef7f70c11e26ac82bc3716d47902ad3a773fa48/db/db_impl/db_impl.cc#L928

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.