deepseek-ai / deepseek-ai/DeepSeek-V2

Proposal: Symbolic Activation — 95% Compression & No Context Rot

Open
#118 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
No language data
Stars
5k
Forks
550
PR merge metrics
No merged PRs in 30d

Description

Hi @guosheng @deepseek-ai,

I'm GCat, and I've published research identifying **Context Rot**—a critical flaw in KV caching that degrades LLM performance in long conversations.

Key findings:
- After 15+ turns / 32k tokens, effective info utilization drops below 40%
- Root cause: Transformer + explicit KV cache, not long-context capability
- Solution: **Symbolic Activation Framework** → 95% compression, zero context pollution

Paper: https://zenodo.org/records/20433184
Code: https://github.com/gymaira1990-jpg/catnest/tree/main/00-%E8%AE%BA%E6%96%87%E5%90%88%E9%9B%86

I would love to hear your thoughts on feasibility of integrating this into DeepSeek inference.

Thanks!
GCat

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.