mlcommons / mlcommons/endpoints
[Feature]: Deepseek R1 reference implementation
Open
@arekay-nv is already working on this.
Since Apr 28, 2026.
status: needs-triage
type: feature
- Dominant language
- Python
- Stars
- 21
- Forks
- 28
- Avg merge
- 3d 17h
- Merged PRs (30d)
- 13
Description
Motivation
A reference implementation for Deepseek-r1. The implementation should include the dataset (and sampled versions if needed for lower concurrency runs), as well as accuracy checks and allowed optimizations.
Proposed Solution
Possibly using the MLPerf inference setup and tweaking for different concurrency points. Concurrency=1 would require more thought.
Alternatives Considered
No response
Additional Context
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.