allenai/dolma

Data and tools for generating and inspecting OLMo pre-training data.

Open All

7 open issues indexed

not_alphanum_paragraph_v1 tagger takes forever to run on certain inputs. Open

allenai/dolma#123 · 1 comment · 0 reactions · 1 assignee · Indexed Sep 13, 2026
Support providing streams into mixer via CLI Open

enhancement

allenai/dolma#130 · 0 comments · 0 reactions · 1 assignee · Indexed Sep 13, 2026
Config files for dolma 1.7 Open

allenai/dolma#255 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
Unclear on dedupe outputs for decontamination Open

allenai/dolma#264 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
Model path cannot be resolved Open

allenai/dolma#268 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 13, 2026
Unlimited Generation During Tokenization Open

allenai/dolma#272 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 13, 2026
https://support.google.com/websearch/thread/215389716?hl=en&msgid=215402255 Open

allenai/dolma#301 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026

Get new issues in your inbox