deepseek-ai / deepseek-ai/DeepSpec

Support online training without precomputing the target cache

Open
#21 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
7.1k
Forks
667
PR merge metrics
No merged PRs in 30d

Description

Hi, per the [data prep docs](https://github.com/deepseek-ai/DeepSpec/blob/main/scripts/data/README.md), training requires an offline step (`prepare_target_cache.py`) that stores the target model's hidden states to disk. The docs warn this cache is _~38 TB_ for `Qwen/Qwen3-4B`, which is a blocker without large-scale storage.

Could you add an online training mode that runs the target model on the fly instead of reading from a precomputed cache? I understand it trades storage for longer training time, but it would help on storage-limited setups. Frameworks like [SpecForge](https://github.com/sgl-project/SpecForge) already support this.

Thanks!

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.