couchbaselabs / couchbaselabs/cbft

pIndex file generated on disk are close to 10x size of the bucket size in memory

Open
#165 6 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Go
Stars
26
Forks
5
PR merge metrics
No merged PRs in 30d

Description

```
➜ data du -sch webnutshell_21809c13557138e3_* | grep total
592M total
➜ data /opt/couchbase/bin/cbstats 0:11210 -b webnutshell all | grep -w
mem_used
mem_used: 66951456
```

3 types of documents in this bucket(webnutshell), redacted some confidential customer info:

cluster_blob: https://gist.github.com/abhi-bit/6bbbcac3ff75d20b0e00
node_blob: https://gist.github.com/abhi-bit/a8892159fd684c510fb6
customer_blob: https://gist.github.com/abhi-bit/62882eae79602bcda77c

Also, I've noticed the ratio of cbft index files vs bucket mem_used to grow as bucket dataset size grows. From an earlier deployment experience, I've seen a bucket using ~1G in memory created indexes of size 190GB on disk - I've kick started indexing against that bucket couple of days back, will share numbers once the indexing is complete there.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.