allegro / allegro/bigcache

Inadequate memory consumption

Aperta
#292 4 commenti 0 reazioni 0 assegnatari Vedi su GitHub
bug
Lingua principale
Go
Stelle
8.2k
Fork
614
Merge medio
5g 12h
PR unite (30g)
1

Descrizione

**What is the issue you are having?**
Despite the limitation of the maximum cache size of 64 megabytes, the program consumes considerably more memory

I'se set HardMaxCacheSize to 64Mb (see a code below), but after programm run vmRSS was much more than 64Mb. In a real setup I've seen cache size more than 5Gb and programm was killed by OOM killer

keys count = 1
vmRSS: Pre run 19 890 176
vmRSS: Post run 20 430 848

keys count = 100
vmRSS: Pre run 19 890 176
vmRSS: Post run 74 498 048

keys count = 1000
vmRSS: Pre run 198 86 080
vmRSS: Post run 385 257 472

keys count = 5000
vmRSS: Pre run 19 894 272
vmRSS: Post run 556 134 400

**What is BigCache doing that it shouldn't?**
Consumes a lot of memory

**Minimal, Complete, and Verifiable Example**

```
package main

import (
"fmt"
"log"
"strconv"
"time"

"github.com/allegro/bigcache/v3"
"github.com/prometheus/procfs"
)

func main() {
config := bigcache.Config{
Shards: 1024,
LifeWindow: 10 * time.Minute,
CleanWindow: 5 * time.Minute,
MaxEntriesInWindow: 5000 * 10 * 60,
MaxEntrySize: 1024,
Verbose: true,
HardMaxCacheSize: 64,
OnRemove: nil,
OnRemoveWithReason: nil,
}

cache, initErr := bigcache.NewBigCache(config)
if initErr != nil {
log.Fatal(initErr)
}

memStat("Pre run")

// Keys count
count := 1000
val := make([]byte, 512*1024)

for count > 0 {
key := fmt.Sprintf("key-%d", count)
cache.Set(key, val)
count--
}

memStat("Post run")
}

func memStat(stage string) {

p, err := procfs.Self()
if err != nil {
return
}

stat, err := p.NewStatus()
if err != nil {
return
}

fmt.Printf("vmRSS: %s %d\n", stage, int64(stat.VmRSS))
}

```
**Environment:**
go version go1.17 linux/amd64

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.