ByteDance-Seed / ByteDance-Seed/ShadowKV
[QUESTION] KV budget for longbench
Open
- Dominant language
- Python
- Stars
- 313
- Forks
- 26
- PR merge metrics
- No merged PRs in 30d
Description
Hi, I have a question regarding the kv budget size used to evaluate longbench.
In the paper, it states '256' budget size is used.
However, I'm not sure whether it means 256 chunks (total of 2048 kv) or 256 kv.
I was confused because, in the case of outlier budget, the paper states that outlier budget was 48, but it was actually 48 * 8 in your code.
What's the exact kv budget size used in longbench evaluation?
Contributor guide
Assessment
This issue has not been assessed yet.