allegro / allegro/bigcache

Inadequate memory consumption

Ouverte
#292 4 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
bug
Langage dominant
Go
Étoiles
8.2k
Forks
614
Merge moyen
5 j 12 h
PR mergées (30 j)
1

Description

**What is the issue you are having?**
Despite the limitation of the maximum cache size of 64 megabytes, the program consumes considerably more memory

I'se set HardMaxCacheSize to 64Mb (see a code below), but after programm run vmRSS was much more than 64Mb. In a real setup I've seen cache size more than 5Gb and programm was killed by OOM killer

keys count = 1
vmRSS: Pre run 19 890 176
vmRSS: Post run 20 430 848

keys count = 100
vmRSS: Pre run 19 890 176
vmRSS: Post run 74 498 048

keys count = 1000
vmRSS: Pre run 198 86 080
vmRSS: Post run 385 257 472

keys count = 5000
vmRSS: Pre run 19 894 272
vmRSS: Post run 556 134 400

**What is BigCache doing that it shouldn't?**
Consumes a lot of memory

**Minimal, Complete, and Verifiable Example**

```
package main

import (
"fmt"
"log"
"strconv"
"time"

"github.com/allegro/bigcache/v3"
"github.com/prometheus/procfs"
)

func main() {
config := bigcache.Config{
Shards: 1024,
LifeWindow: 10 * time.Minute,
CleanWindow: 5 * time.Minute,
MaxEntriesInWindow: 5000 * 10 * 60,
MaxEntrySize: 1024,
Verbose: true,
HardMaxCacheSize: 64,
OnRemove: nil,
OnRemoveWithReason: nil,
}

cache, initErr := bigcache.NewBigCache(config)
if initErr != nil {
log.Fatal(initErr)
}

memStat("Pre run")

// Keys count
count := 1000
val := make([]byte, 512*1024)

for count > 0 {
key := fmt.Sprintf("key-%d", count)
cache.Set(key, val)
count--
}

memStat("Post run")
}

func memStat(stage string) {

p, err := procfs.Self()
if err != nil {
return
}

stat, err := p.NewStatus()
if err != nil {
return
}

fmt.Printf("vmRSS: %s %d\n", stage, int64(stat.VmRSS))
}

```
**Environment:**
go version go1.17 linux/amd64

Guide de contribution

Aucun guide de contribution indexé pour ce dépôt

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.