mlfoundations/open_lm

View on GitHub

A repository for research on medium sized language models.

Stars: 536
Forks: 80
Open beginner issues: 0
Indexed issues: 35

Dominant language: Python
License: MIT
Last GitHub push: Jul 23, 2024
Latest indexed: Sep 19, 2026
Contributing guide: No contributing guide
Code of conduct: No code of conduct
Beginner labels: good first issue
PR merge metrics: No merged PRs in 30d

Beginner friendly Everything

Open All

35 open issues indexed

Is there a problem with sample loading? Open

Difficulty 1/5 Under an hour Newbie friendliness 30/100

mlfoundations/open_lm#308 · Opened Jul 18, 2025
training loss not decreased Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

mlfoundations/open_lm#307 · Opened May 29, 2025
How to use multi source manifest data with different size balancedly? Open

Difficulty 5/5 Over a week Newbie friendliness 15/100

mlfoundations/open_lm#306 · 1 reaction · Opened Sep 6, 2024
Multi-node training Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

mlfoundations/open_lm#305 · 1 comment · Opened Aug 31, 2024
How to pretrain on DCLM-BASELINE Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

mlfoundations/open_lm#304 · 1 reaction · Opened Aug 30, 2024
NotImplementedError running HF model "mlfoundations/dclm-7b-it" for inference Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

mlfoundations/open_lm#303 · 1 comment · Opened Aug 27, 2024
Webdataset version issue Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

mlfoundations/open_lm#301 · 2 reactions · Opened Aug 21, 2024
I got an error in open lm installation Open

Difficulty 3/5 1-2 days Newbie friendliness 25/100

mlfoundations/open_lm#297 · 2 comments · Opened Jul 27, 2024
Fine-Tuned Models for open_lm Open

Difficulty 5/5 Over a week Newbie friendliness 15/100

mlfoundations/open_lm#296 · Opened Jul 25, 2024
Remote Sync FSSPEC cannot upload large checkpoints Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

mlfoundations/open_lm#279 · Opened May 18, 2024
samples_per_second_per_gpu or tokens_per_second_per_gpu? Open

Difficulty 2/5 1-3 hours Newbie friendliness 45/100

mlfoundations/open_lm#262 · 1 comment · Opened Apr 29, 2024
MoE Expert parallelism config Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

mlfoundations/open_lm#251 · Opened Apr 15, 2024
Someone is using your project to sell it as a token Open

Difficulty 5/5 Over a week Newbie friendliness 10/100

mlfoundations/open_lm#247 · 1 comment · 1 reaction · Opened Apr 12, 2024
Support user specified token pre-processing functions Open

enhancement good first issue

Difficulty 5/5 Over a week Newbie friendliness 25/100

mlfoundations/open_lm#194 · Opened Jan 7, 2024
Factorize helper function for all model loading Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

mlfoundations/open_lm#181 · Opened Dec 23, 2023
grad accum tests failing on gpu w/ amp_bf16 precision Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

mlfoundations/open_lm#171 · Opened Dec 19, 2023
Undefined argument "moe_freq" when running unit tests on WSL/Ubuntu 20.04 Open

mlfoundations/open_lm#168 · 1 comment · 1 assignee · Opened Dec 19, 2023
`--delete-previous-checkpoint` should delete prev checkpoints in `--remote-sync` bucket Open

good first issue

Difficulty 3/5 1-2 days Newbie friendliness 42/100

mlfoundations/open_lm#166 · Opened Dec 19, 2023
Error early if we don't have enough disk space Open

good first issue

Difficulty 5/5 Over a week Newbie friendliness 35/100

mlfoundations/open_lm#154 · Opened Dec 14, 2023
Investigate effect of FSDP policies on mamba speed Open

Difficulty 5/5 Over a week Newbie friendliness 15/100

mlfoundations/open_lm#144 · Opened Dec 9, 2023
Support saving/loading models larger than CPU memory Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

mlfoundations/open_lm#130 · Opened Dec 4, 2023
Figure out why AdamW + gradient accumulation leads to different results for test case Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

mlfoundations/open_lm#126 · 6 comments · Opened Nov 30, 2023
Move dummy cred download into test Open

mlfoundations/open_lm#121 · 1 assignee · Opened Nov 30, 2023
clean up model_configs directory Open

mlfoundations/open_lm#116 · 1 assignee · Opened Nov 27, 2023
Add venv cache for CI to avoid installing dependencies Open

good first issue

Difficulty 2/5 1-3 hours Newbie friendliness 48/100

mlfoundations/open_lm#104 · Opened Nov 21, 2023
open_lm chronicles Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 25/100

mlfoundations/open_lm#90 · Opened Nov 14, 2023
HF Integration Open

enhancement

mlfoundations/open_lm#89 · 3 comments · 1 assignee · Opened Nov 14, 2023
Minimize how often we load args.resume Open

enhancement good first issue

Difficulty 2/5 1-3 hours Newbie friendliness 48/100

mlfoundations/open_lm#71 · 1 comment · Opened Nov 6, 2023
Tokenization on-the-fly without slowdown Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

mlfoundations/open_lm#55 · Opened Oct 24, 2023
llama2 unit tests Open

good first issue

Difficulty 3/5 1-2 days Newbie friendliness 45/100

mlfoundations/open_lm#52 · Opened Oct 22, 2023
Documentation: competing frameworks Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

mlfoundations/open_lm#41 · 3 comments · Opened Oct 18, 2023
fp8 training on h100s Open

Difficulty 5/5 Over a week Newbie friendliness 15/100

mlfoundations/open_lm#37 · Opened Oct 16, 2023
Weird memory usage for 11m vs 160m: similar batch size fits in memory... Open

Difficulty 4/5 3-5 days Newbie friendliness 32/100

mlfoundations/open_lm#35 · 14 comments · 1 reaction · Opened Oct 14, 2023
Benchmark tok/sec with other libs Open

mlfoundations/open_lm#21 · 4 comments · 3 reactions · 1 assignee · Opened Sep 26, 2023
Release the intermediate checkpoints? Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

mlfoundations/open_lm#19 · 2 comments · Opened Sep 26, 2023

mlfoundations/open_lm

Get new issues in your inbox