huggingface/datablations

Scaling Data-Constrained Language Models

Stars: 345
Forks: 19
Open beginner issues: 0
Indexed issues: 4

Dominant language: Jupyter Notebook
License: Apache-2.0
Last GitHub push: Jun 28, 2025
Latest indexed: Sep 17, 2026
Contributing guide: No contributing guide
Code of conduct: No code of conduct
Beginner labels: No beginner labels indexed
PR merge metrics: No merged PRs in 30d

Open All

4 open issues indexed

A question about the conclusion of this paper Open

huggingface/datablations#6 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Figure issue about your paper (Figure 4 and Figure 15) Open

huggingface/datablations#7 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
wonder if LR=1e-3 for mup is optimal value from small-scale proxy model and dropout is crucial for multi-epoch Open

huggingface/datablations#13 · 4 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Validation loss vs model size per step Open

huggingface/datablations#14 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026

A short digest of beginner-friendly GitHub issues.