deepseek-ai / deepseek-ai/Engram

A Use Case in Rust

Open
#8 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
4.7k
Forks
359
PR merge metrics
No merged PRs in 30d

Description

Thanks for publishing this, very interesting optimization path.

I spend some of my time with the [attention.rs](https://github.com/guoqingbao/attention.rs) project (authored and maintained by @guoqingbao, partially to support domestically designed/produced kit in an API-compatible manner with global market fare) which already underpins at least two inference libraries and has broader application. If the DeepSeek team happens to have any resources available to implement this demo in a more structured domain, it could do a lot of good applied to an attention/KV/memory dependency in the Rust ML ecosystem; especially for folks trying to work with large MoEs like DeepSeek on somewhat limited/partially offloaded HW (your team makes some "parameter-heavy" models which require a ton of VRAM in conventional use - not the sort of kit to which most ML practitioners or students have access).

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.