[BUG]: Badger expired data is still there(using txn.Get(k) ). after a restart or several days running.

What version of Badger are you using?

github.com/dgraph-io/badger/v4 v4.2.0

What version of Go are you using?

go 1.22.0
toolchain go1.22.4

Have you tried reproducing the issue with the latest release?

No

What is the hardware spec (RAM, CPU, OS)?

16Core CPU. 64GB Ram , Linux

What steps will reproduce the bug?

I am not completely sure about how to reproduce the bug. It will appear after I run several hours or days badger operation. After a restart, some of the expired keys is still there when I scan the related keys or get the key directly.

Can someone give me a bit of advice or guidance? Thanks.

Pseudocode：

func writeData(k1, k2, k3[]byte) {
  	txn := tu.db.NewTransaction(true)
	defer txn.Discard()

        k1DataV1 := txn.Get(k1)
        k1DataV2 := processK1(k1DataV1)
        txn.set(k1, k1DataV2)

        k2DataV1 := txn.Get(k2)
        k2DataV2 := processK1(k2DataV1)
        txn.set(k2, k2DataV2)

        k3DataV1 := txn.Get(k3)
        k3DataV2 := processK1(k3DataV1)
        txn.set(k3, k3DataV2)

        txn.setWithTTL(k2, k2DataV3, time.Second * 300)

}

Expected behavior and actual result.

Expired data should not be accessable via the basic get or scan APIs of badgerdb.
Should return key not found like errors.

Additional information

shutdown is called when the program caught kill signal.

The sync error log didn't appear.
The Server exiting end --- log appeared.

shutdown code

func BadgerClose() {
	for _, v := range defaultMgr.instances {
		if v != nil {
			if syncErr := v.Sync(); syncErr != nil {
				logger.Errorf("sync error:%+v\n", syncErr)
			}
			err := v.Close()
			if err != nil {
				logger.Errorln(err.Error())
			}
		}
	}
}

func shutdown() {
     BadgerClose()
     logger.Info("Server exiting end ---------------- ")
}

badger options

opts := badger.DefaultOptions(getPath(badgerCfg.BasePath, key))
opts = opts.WithLogger(logger)
opts = opts.WithMetricsEnabled(true)
opts = opts.WithNumMemtables(badgerCfg.MemTableNum)
opts = opts.WithMemTableSize(badgerCfg.MemTableSize << 20)
opts = opts.WithNumLevelZeroTables(8)
opts = opts.WithNumCompactors(4)
opts = opts.WithValueLogFileSize(128 << 20)
opts = opts.WithCompactL0OnClose(true)
db, err := badger.Open(opts)

Start with the pseudocode using txn.Get, txn.SetWithTTL, and the scan/get behavior, then review the BadgerClose shutdown path and the listed options, including WithCompactL0OnClose. Reproduce the report on the latest release across a restart and several days of operation. Done means expired keys are inaccessible through basic get and scan APIs, with the behavior covered by a regression test.