google / google/leveldb

My implementation: Compaction IO speed limitation

Open
#185 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
C++
Stars
39.4k
Forks
8.2k
PR merge metrics
No merged PRs in 30d

Description

Original [issue 179](https://code.google.com/p/leveldb/issues/detail?id=179) created by wuzuyang on 2013-06-15T05:28:08.000Z:

Hi, everyone:

I created an issue days ago(https://code.google.com/p/leveldb/issues/detail?id=168&sort=-id), to require the feature of compaction IO speed limitaion. But no reponse.

As we know, the leveldb has extremly BAD performance during compaction, because it eats up all the disk IO capacity. So a compaction IO speed limitation feature is very important.

I wrote my own. Here is some of the codes:

``` patch
diff --git a/deps/leveldb-1.9.0/db/db_impl.cc b/deps/leveldb-1.9.0/db/db_impl.cc
index c9de169..56915f4 100644
--- a/deps/leveldb-1.9.0/db/db_impl.cc
+++ b/deps/leveldb-1.9.0/db/db_impl.cc
@@ -805,6 +805,22 @@ Status DBImpl::FinishCompactionOutputFile(CompactionState* compact,
(unsigned long long) output_number,
(unsigned long long) current_entries,
(unsigned long long) current_bytes);
+
+ // Writing current_bytes to disk is considered no expense(cost no time),
+ // so we calculate how many IOs will match the compaction speed,
+ // then sleep 1s/IOs
+ // Added by me@ideawu.com
+ int mbs = current_bytes/1024/1024;
+ if(options_.compaction_speed > 0 && mbs > 1){
+ int count = options_.compaction_speed/mbs;
+ if(count < 1){
+ count = 1;
+ }
+ int pause = 1000 * 1000 / count;
+ Log(options_.info_log, "compaction_speed: %d MB, pause: %d us",
+ options_.compaction_speed, pause);
+ env_->SleepForMicroseconds(pause);
+ }
}
}
return s;
```

This is a very primative approach, I am expecting there is an official implementation.

The full codes is here: https://github.com/ideawu/ssdb/tree/master/deps/leveldb-1.9.0

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.