couchbaselabs / couchbaselabs/cbft

storage config params dynamically calculated

Open
#30 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
26
Forks
5
PR merge metrics
No merged PRs in 30d

Description

As more nodes get added to a cbft cluster, the feature idea is to have some config settings dynamically dependent on how many pindexes a node is serving (so, based on how much mem is available, etc).

---

From some research from Marty Schoch...

I played around with some different leveldb configs with bleve-bench. It looks like the following config works best:

{
"write_buffer_size": 536870912,
"lru_cache_capacity": 536870912,
"bloom_filter_bits_per_key": 10
}

In my tests, the indexing in batches is almost 2x as fast with these settings. Unfortunately, none of these settings seems to affect the query speed very much. ... Still investigating…

You may want to try these settings on intrapop and see if they help.

Oh, unfortunately unlike ForestDB, with LevelDB the configuration is at the database level. Above, the 2 caches are 512MB each. So, if your number of shards \* 1GB is greater than the available ram, you should probably reduce them.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.