deepseek-ai / deepseek-ai/DeepSeek-Coder
How to extended window size during train step2?
Open
- Dominant language
- Python
- Stars
- 24.3k
- Forks
- 2.9k
- PR merge metrics
- No merged PRs in 30d
Description
`https://github.com/deepseek-ai/DeepSeek-Coder#model-training` descibe "Further Pre-training using an extended 16K window size on an additional 200B tokens", How to extend the window size during training? Just modify max_length and max_position_embeddings in the “config.json” file? or what I need to do?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.