allegro / allegro/bigcache

Inadequate memory consumption

Offen
#292 4 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
bug
Vorherrschende Sprache
Go
Sterne
8.2k
Forks
614
Ø Merge
5 T. 12 Std.
Gemergte PRs (30 T.)
1

Beschreibung

**What is the issue you are having?**
Despite the limitation of the maximum cache size of 64 megabytes, the program consumes considerably more memory

I'se set HardMaxCacheSize to 64Mb (see a code below), but after programm run vmRSS was much more than 64Mb. In a real setup I've seen cache size more than 5Gb and programm was killed by OOM killer

keys count = 1
vmRSS: Pre run 19 890 176
vmRSS: Post run 20 430 848

keys count = 100
vmRSS: Pre run 19 890 176
vmRSS: Post run 74 498 048

keys count = 1000
vmRSS: Pre run 198 86 080
vmRSS: Post run 385 257 472

keys count = 5000
vmRSS: Pre run 19 894 272
vmRSS: Post run 556 134 400

**What is BigCache doing that it shouldn't?**
Consumes a lot of memory

**Minimal, Complete, and Verifiable Example**

```
package main

import (
"fmt"
"log"
"strconv"
"time"

"github.com/allegro/bigcache/v3"
"github.com/prometheus/procfs"
)

func main() {
config := bigcache.Config{
Shards: 1024,
LifeWindow: 10 * time.Minute,
CleanWindow: 5 * time.Minute,
MaxEntriesInWindow: 5000 * 10 * 60,
MaxEntrySize: 1024,
Verbose: true,
HardMaxCacheSize: 64,
OnRemove: nil,
OnRemoveWithReason: nil,
}

cache, initErr := bigcache.NewBigCache(config)
if initErr != nil {
log.Fatal(initErr)
}

memStat("Pre run")

// Keys count
count := 1000
val := make([]byte, 512*1024)

for count > 0 {
key := fmt.Sprintf("key-%d", count)
cache.Set(key, val)
count--
}

memStat("Post run")
}

func memStat(stage string) {

p, err := procfs.Self()
if err != nil {
return
}

stat, err := p.NewStatus()
if err != nil {
return
}

fmt.Printf("vmRSS: %s %d\n", stage, int64(stat.VmRSS))
}

```
**Environment:**
go version go1.17 linux/amd64

Beitragsleitfaden

Für dieses Repository ist kein Beitragsleitfaden indexiert

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.