allegro / allegro/bigcache

Inadequate memory consumption

Abierto
#292 4 comentarios 0 reacciones 0 asignados Ver en GitHub
bug
Lenguaje dominante
Go
Estrellas
8.2k
Forks
614
Merge medio
5 d 12 h
PR fusionados (30 d)
1

Descripción

**What is the issue you are having?**
Despite the limitation of the maximum cache size of 64 megabytes, the program consumes considerably more memory

I'se set HardMaxCacheSize to 64Mb (see a code below), but after programm run vmRSS was much more than 64Mb. In a real setup I've seen cache size more than 5Gb and programm was killed by OOM killer

keys count = 1
vmRSS: Pre run 19 890 176
vmRSS: Post run 20 430 848

keys count = 100
vmRSS: Pre run 19 890 176
vmRSS: Post run 74 498 048

keys count = 1000
vmRSS: Pre run 198 86 080
vmRSS: Post run 385 257 472

keys count = 5000
vmRSS: Pre run 19 894 272
vmRSS: Post run 556 134 400

**What is BigCache doing that it shouldn't?**
Consumes a lot of memory

**Minimal, Complete, and Verifiable Example**

```
package main

import (
"fmt"
"log"
"strconv"
"time"

"github.com/allegro/bigcache/v3"
"github.com/prometheus/procfs"
)

func main() {
config := bigcache.Config{
Shards: 1024,
LifeWindow: 10 * time.Minute,
CleanWindow: 5 * time.Minute,
MaxEntriesInWindow: 5000 * 10 * 60,
MaxEntrySize: 1024,
Verbose: true,
HardMaxCacheSize: 64,
OnRemove: nil,
OnRemoveWithReason: nil,
}

cache, initErr := bigcache.NewBigCache(config)
if initErr != nil {
log.Fatal(initErr)
}

memStat("Pre run")

// Keys count
count := 1000
val := make([]byte, 512*1024)

for count > 0 {
key := fmt.Sprintf("key-%d", count)
cache.Set(key, val)
count--
}

memStat("Post run")
}

func memStat(stage string) {

p, err := procfs.Self()
if err != nil {
return
}

stat, err := p.NewStatus()
if err != nil {
return
}

fmt.Printf("vmRSS: %s %d\n", stage, int64(stat.VmRSS))
}

```
**Environment:**
go version go1.17 linux/amd64

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.