GC doesn't seem to run
Open
Nobody has claimed this yet.
kind/bug
Stale
- Dominant language
- Go
- Stars
- 15.8k
- Forks
- 1.3k
- Avg merge
- 2m
- Merged PRs (30d)
- 1
Description
What version of Badger are you using?
Latest v4
What version of Go are you using?
1.20.3
Have you tried reproducing the issue with the latest release?
Yes
What is the hardware spec (RAM, CPU, OS)?
16gb ram, i5 intel, mac os
What steps will reproduce the bug?
package main
import (
"fmt"
"log"
"math/rand"
"strconv"
"time"
"github.com/dgraph-io/badger/v4"
)
var db *badger.DB
const charset = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"
func generateRandomString(length int) string {
b := make([]byte, length)
for i := range b {
b[i] = charset[rand.Intn(len(charset))]
}
return string(b)
}
func main() {
opts := badger.DefaultOptions("badger")
opts.ValueLogFileSize = 1 << 20
var err error
db, err = badger.Open(opts)
if err != nil {
log.Fatal(err)
}
defer db.Close()
fmt.Println("Start inserting 5 million items.")
for i := 0; i < 5_000_000; i++ {
err := db.Update(func(txn *badger.Txn) error {
key := strconv.Itoa(i)
value := []byte(generateRandomString(300))
e := badger.NewEntry([]byte(key), value).WithTTL(30 * time.Second)
err := txn.SetEntry(e)
if err != nil {
return err
}
return nil
})
if err != nil {
log.Fatal("Error while inserting:", err)
}
}
fmt.Println("Successfully inserted 5 million items.")
startGarbageCollection()
}
func startGarbageCollection() {
ticker := time.NewTicker(10 * time.Second)
defer ticker.Stop()
for range ticker.C {
again:
err := db.RunValueLogGC(0.01)
fmt.Println(err)
if err == nil {
goto again
}
}
}
Expected behavior and actual result.
Since the keys are expiring in 30 seconds, I thought that the files in badger/ would get removed.
However, heres what I see:
Additional information
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the provided Go reproducer and the db.RunValueLogGC entry point, then compare the observed retention of expired entries and value-log files with Badger's documented GC behavior. Done means determining whether the result is expected or defective and, if defective, identifying a focused fix and test for the reported case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 42/100