kvcache-ai / kvcache-ai/Mooncake
[RFC]: Reproducing Table 1 hit rates on the released trace, consistent +4-6pp offset
- Dominant language
- C++
- Stars
- 6.6k
- Forks
- 1.2k
- Avg merge
- 3d 5h
- Merged PRs (30d)
- 312
Description
### Changes proposed
I've been reproducing your Table 1 cache hit rates on arxiv-trace/mooncake_trace.jsonl using LRU, and I get the same curve shape but a systematic offset at every capacity:
| blocks | 1k | 10k | 30k | 50k | 100k | infiniti |
|--------|-------|-------|-------|-------|-------|-------|
| paper | 0.30 | 0.40 | 0.48 | 0.50 | 0.51 | 0.51 |
| mine | 0.341 | 0.460 | 0.537 | 0.551 | 0.552 | 0.553 |
I compute hit rate as (sum of longest-resident-prefix blocks) / (sum of all blocks requested), replaying in timestamp order. Since the infinite-cache case is policy-free, I'd expect it to be reproducible from the trace alone, but I get 0.553 against your 0.51.
I've tried five denominator definitions (blocks vs input_length tokens, excluding the partial tail block, per-request averaging) and none closes the gap. Is the released trace the same one used for Table 1, or is there a detail in the hit-rate definition I'm missing?
Repro: https://github.com/gauravapiscean/agentic-kv-cache
### Before submitting a new issue...
- [x] Make sure you already searched for relevant issues and read the [documentation](https://kvcache-ai.github.io/Mooncake/)
Contributor guide
Assessment
This issue has not been assessed yet.