mlcommons/algorithmic-efficiency

View on GitHub

MLCommons Algorithmic Efficiency is a benchmark and competition measuring neural network training speedups due to algorithmic improvements in both training algorithms and models.

Stars: 425
Forks: 78
Open beginner issues: 0
Indexed issues: 31

Dominant language: Python
License: Apache-2.0
Last GitHub push: Apr 3, 2026
Latest indexed: Sep 19, 2026
Contributing guide: Contributing guide
Code of conduct: No code of conduct
Beginner labels: good first issue
PR merge metrics: No merged PRs in 30d

Beginner friendly Everything

Open All

31 open issues indexed

schedule-free adamw: Analyze why jax performed better than pytorch on finewebedu workload Open

Difficulty 5/5 Over a week Newbie friendliness 30/100

mlcommons/algorithmic-efficiency#925 · Opened May 19, 2026
librispeech deepspeech validation loss differs between pytorch and jax, but wer aligns Open

Difficulty 4/5 3-5 days Newbie friendliness 38/100

mlcommons/algorithmic-efficiency#924 · 5 comments · Opened May 9, 2026
Anima: live PH monitoring in conversational agent — overfitting detection every 50 interactions Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

mlcommons/algorithmic-efficiency#919 · Opened Mar 25, 2026
Dual-engine architecture: tension-based confidence yields AUROC=1.0 anomaly detection and AUC=0.925 precognition Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

mlcommons/algorithmic-efficiency#918 · Opened Mar 25, 2026
Topological overfitting detection: H0 gap catches overfitting before accuracy diverges (r=0.998) Open

Difficulty 5/5 Over a week Newbie friendliness 15/100

mlcommons/algorithmic-efficiency#917 · Opened Mar 25, 2026
Add step_hint_batch_size property to workloads to allow submissions to know the batch size the baseline defining the step hint used Open

Difficulty 3/5 1-2 days Newbie friendliness 48/100

mlcommons/algorithmic-efficiency#914 · Opened Feb 27, 2026
Cifar dataloader does not work Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

mlcommons/algorithmic-efficiency#889 · 2 comments · Opened Sep 18, 2025
Add Lion baseline Open

⚡ Submission

mlcommons/algorithmic-efficiency#879 · 2 comments · 1 assignee · Opened Aug 5, 2025
Speech workloads speed regression on JAX Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

mlcommons/algorithmic-efficiency#877 · Opened Jul 15, 2025
Publish Docker Image to Docker Hub to Support a Wider Set of Contributors Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

mlcommons/algorithmic-efficiency#868 · Opened Apr 22, 2025
v2 Self-Tuning Budget for ResNet is Half of Benchmark Schedule-Free Run Time Open

Difficulty 3/5 1-2 days Newbie friendliness 45/100

mlcommons/algorithmic-efficiency#867 · 1 comment · Opened Apr 19, 2025
submission_runner_test.py is not included in CI tests Open

🧪 Test

Difficulty 2/5 1-3 hours Newbie friendliness 45/100

mlcommons/algorithmic-efficiency#865 · Opened Apr 17, 2025
Recalibrate eval time intervals Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

mlcommons/algorithmic-efficiency#858 · Opened Apr 2, 2025
Optimize Git Hook Configuration Open

Good First Issue

Difficulty 3/5 1-2 days Newbie friendliness 35/100

mlcommons/algorithmic-efficiency#853 · 1 comment · Opened Mar 13, 2025
Add schedule-free adamw submission in JAX Open

⚡ Submission

mlcommons/algorithmic-efficiency#809 · 20 comments · 1 assignee · Opened Oct 31, 2024
Clean up train diff tests Open

Good First Issue

Difficulty 3/5 1-2 days Newbie friendliness 45/100

mlcommons/algorithmic-efficiency#808 · Opened Oct 31, 2024
Wrong return type in librispeech model_fn Open

Difficulty 1/5 Under an hour Newbie friendliness 48/100

mlcommons/algorithmic-efficiency#762 · Opened May 9, 2024
Example fails at torch.compile Open

Difficulty 4/5 3-5 days Newbie friendliness 38/100

mlcommons/algorithmic-efficiency#754 · Opened Apr 5, 2024
Resnet DDP Warning: grad strides do not match bucket view strides Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

mlcommons/algorithmic-efficiency#741 · 3 comments · Opened Mar 29, 2024
Incorrect Imagenet evals with pytorch_eval_num_workers > 0 Open

🐛 Bug 🔥 PyTorch

Difficulty 4/5 3-5 days Newbie friendliness 25/100

mlcommons/algorithmic-efficiency#732 · 2 comments · Opened Mar 27, 2024
Skip eval on train and test for self-reporting results Open

✨ Feature Request Good First Issue

Difficulty 4/5 3-5 days Newbie friendliness 38/100

mlcommons/algorithmic-efficiency#725 · 2 comments · Opened Mar 26, 2024
Two copies of `criteo_resnet_pytorch` exist in .github/workflows/regression_tests_variants.yml Open

Difficulty 2/5 1-3 hours Newbie friendliness 45/100

mlcommons/algorithmic-efficiency#704 · 1 comment · Opened Mar 12, 2024
LibriSpeech Conformer Workload OOMs or NCCL Errors When Run With Multiple Trials Open

🔥 PyTorch

Difficulty 4/5 3-5 days Newbie friendliness 35/100

mlcommons/algorithmic-efficiency#663 · 9 comments · Opened Feb 29, 2024
Add tests for scoring code Open

Difficulty 3/5 1-2 days Newbie friendliness 48/100

mlcommons/algorithmic-efficiency#624 · Opened Feb 1, 2024
LM1B: Jax Open

✨ Feature Request Future Version

Difficulty 5/5 Over a week Newbie friendliness 25/100

mlcommons/algorithmic-efficiency#621 · Opened Jan 24, 2024
Add dataset setup tests Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

mlcommons/algorithmic-efficiency#486 · Opened Aug 14, 2023
Add tests for expected number of evals submission and measured times Open

Good First Issue

mlcommons/algorithmic-efficiency#452 · 1 assignee · Opened Jul 25, 2023
Add non-baseline example submissions Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

mlcommons/algorithmic-efficiency#309 · Opened Feb 3, 2023
rewrite halton.py to use the scipy quasirandom sequence generator Open

Difficulty 3/5 1-2 days Newbie friendliness 42/100

mlcommons/algorithmic-efficiency#286 · Opened Jan 19, 2023
Split out train and update_batch_norm in Librispeech workloads Open

Difficulty 3/5 1-2 days Newbie friendliness 45/100

mlcommons/algorithmic-efficiency#163 · Opened Oct 8, 2022
Test resetting model parameters via `workload.init_model_fn()` during training Open

mlcommons/algorithmic-efficiency#48 · 1 assignee · Opened Jan 19, 2022

mlcommons/algorithmic-efficiency

Get new issues in your inbox