EleutherAI/pythia

View on GitHub

The hub for EleutherAI's work on interpretability and learning dynamics

Stars: 2.9k
Forks: 224
Open beginner issues: 0
Indexed issues: 23

Dominant language: Jupyter Notebook
License: Apache-2.0
Last GitHub push: Nov 15, 2025
Latest indexed: Sep 15, 2026
Contributing guide: No contributing guide
Code of conduct: No code of conduct
Beginner labels: No beginner labels indexed
PR merge metrics: No merged PRs in 30d

Open All

23 open issues indexed

Reading data is slowly！ Open

EleutherAI/pythia#126 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Wrong files in eval? Open

EleutherAI/pythia#139 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Weird inconsistency in Tokenizer vocabulary Open

EleutherAI/pythia#151 · 1 comment · 0 reactions · 1 assignee · Indexed Sep 15, 2026
Optimizer states in HF format Open

EleutherAI/pythia#152 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Provide the shuffled index_mapping npy files for ease of reproducing training data Open

EleutherAI/pythia#153 · 1 comment · 3 reactions · 0 assignees · Indexed Sep 15, 2026
instruct-tuned pythia Open

EleutherAI/pythia#155 · 0 comments · 1 reaction · 0 assignees · Indexed Sep 15, 2026
Convert to GGUF Open

EleutherAI/pythia#159 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
How to use the Huggingface dataset /EleutherAI/pythia-memorized-evals in predictable-memorization? Open

EleutherAI/pythia#164 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Questions regarding the WSC evaluation results Open

EleutherAI/pythia#172 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
No EOD Tokens in EleutherAI/pile-deduped-pythia-preshuffled Open

EleutherAI/pythia#175 · 2 comments · 1 reaction · 0 assignees · Indexed Sep 15, 2026
Pythia 160M is giving unreasonable logit values Open

EleutherAI/pythia#177 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
`torch.concat` is supported when reproducing results with docker Open

EleutherAI/pythia#178 · 0 comments · 2 reactions · 0 assignees · Indexed Sep 15, 2026
Shard hashes for `EleutherAI/pile-deduped-pythia-preshuffled` Open

EleutherAI/pythia#183 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
seed2 shuffle_idx.npy corrupted Open

EleutherAI/pythia#190 · 1 comment · 1 reaction · 0 assignees · Indexed Sep 15, 2026
using madgraph to study processes at the EP collider with Pythia8 and Delphes enabled Open

EleutherAI/pythia#192 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
How does one obtain the loss for each model, size and checkpoint in the table in the main paper? Open

EleutherAI/pythia#194 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Pythia-2.8B PT CheckPoints is same between steps 0 to steps 6300 Open

EleutherAI/pythia#195 · 3 comments · 3 reactions · 0 assignees · Indexed Sep 15, 2026
Same weights between steps 0 and 2 Open

EleutherAI/pythia#198 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
question on overfitting Open

EleutherAI/pythia#201 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Three missing checkpoints: pythia-410m-seed9@step40000, pythia-410m-seed6@step88000, pythia-410m-seed2@step111000 Open

EleutherAI/pythia#202 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
pythia-160m-weight-seed{1,2} step>=1 checkpoints appear inconsistent with their own step0 initialization Open

EleutherAI/pythia#203 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
pythia-2.8b: 86 of 154 step branches (incl. step0 and the entire early grid) serve final-era checkpoint content Open

EleutherAI/pythia#205 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
pythia-12b: step50000 and step58000 serve the final checkpoint's first shard (embed_in + early layers) Open

EleutherAI/pythia#206 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026

EleutherAI/pythia

Get new issues in your inbox