Dense Olmo 3 7B Dolci-Think 1-epoch SFT as an infra check against the released Think-SFT

Claude Code · https://claude.ai/code/session_01RKJs8PPByCsRbgCucpXmLq — agent-drafted from Abhishek's request in olmo3-think-sft-replication (#1875); for a new session to pick up.

## Goal

Reproduce the paper's Olmo 3 7B Think SFT stage with open-instruct's olmo-core path and check that the result lands near the **released** `allenai/Olmo-3-7B-Think-SFT` under one fixed eval protocol. **Run 1 epoch, not the paper's 2** (Abhishek, 2026-09-09): on the KDA proxy the second epoch was flat on 11/12 dev tasks (#1875), so 1 epoch is the cost-efficient infra check; the released model's extra epoch is a known, bounded difference to state in the comparison. This is an infrastructure check ("does our SFT pipeline still reproduce a known model"), not a research question. The proxy work in #1875 exercised the same code path on a 1.2B latent-KDA MoE; this is the dense 7B counterpart.

## Recipe (from `scripts/train/olmo3/7b_think_sft.sh`, the paper's run)

| knob | paper | notes |
|---|---|---|
| base | Olmo 3 7B long-context base | the weka path in the script is gone; use `allenai/Olmo-3-1025-7B` (HF, in the shared cache at `/weka/oe-eval-default/oyvindt/hf-cache`) |
| data | Dolci-Think SFT mix (`reasoning-mix-decontam-v2-special-tokens-v3-think-FIX`) | that tokenized dir is gone too; tokenize `allenai/Dolci-Think-SFT` with open-instruct, **using the instruct-dev template** (`allenai/olmo-3-tokenizer-instruct-dev`) per `docs/olmo3.md` so `` is not masked out |
| LR | 5e-5 | linear, warmup 3%, anneal to 0 |
| epochs | 2 in the paper; **run 1** | ~24.7B tokens per epoch |
| seq | 32768 | paper; 65536 also fits the base's YaRN context |
| global batch | 1,048,576 tokens | keep the `SEQ × ranks × grad_accum` invariant |
| GPUs | 8×8 in the paper | 2×8 B300 on holmes is the realistic footprint here |

Starting point in the repo: `scripts/train/debug/oc_sft_olmo3_7b_full.sh` (2-epoch olmo-core run, `--rope_scaling_factor 8`, cp_degree 2, mixer `allenai/Dolci-Instruct-SFT`). Swap the model to the base, the mixer to Dolci-Think, LR to 5e-5, tokenizer as above.

## Cost

The earlier dense think probe (W&B `0esmwehu`) did 1.81B tokens in 10.9 h on 8 GPUs, i.e. ~46K tokens/s. One epoch (24.7B tokens) is **roughly 600–1,200 GPU-h** depending on B300 vs H100 and seq length, plus ~50 GPU-h of evals. Budget it as a 2–3 day 16-GPU run.

## Evaluation

Use the paper protocol reproduced in olmo-eval (`abhishekr/paper-protocol`, `strip_thinking` upstreaming as allenai/olmo-eval#361): zero-shot chat, T=0.6 / top-p 0.95, 32K cap, think trace stripped; `minerva_math_:olmo3adapt` ×7, `ifeval`, `ifeval_ood`, `popqa:chat` (limit 2000), `aime_2024:olmo3adapt`. Launcher: `/weka/oe-adapt-default/abhishekr/checkpoints/olmoe3-kda-1.2b-dolci-think-sft-65536/tools/eval_paper_dense.sh` (dense stack with the YaRN `hf_overrides`). Evaluate the final checkpoint with the **think-dev** template (`allenai/olmo-3.2-tokenizer-think-dev`, opens ``), matching how the released Think-SFT is served.

Reference numbers for the released `Olmo-3-7B-Think-SFT` and the `Olmo-3-1025-7B` base under this exact protocol are being measured now and will be posted on #1875 (runs `lift-thinksft-*`, `lift-densebase-*`). Paper table values: MATH 94.4, IFEval 77.9, IFBench 30.0, AIME24 69.6, PopQA 20.8.

**Pass criterion:** final checkpoint within the checkpoint-to-checkpoint noise of the released Think-SFT on MATH / IFEval / IFBench under the same protocol (±2pp was the floor measured on the proxy; measure it here with two seeds if budget allows, otherwise state single-run). Report cap-hit rates alongside scores.

## Prior dense Think runs

Searched issues, W&B (`ai2-llm/open_instruct_internal`), Beaker and the scratch checkpoint dirs on 2026-09-09: the only dense Olmo 3 7B Dolci-Think SFT is W&B `0esmwehu` (scratch dir `m2n5krg1`, 2026-08-14): `allenai/Olmo-3-1025-7B`, Dolci-Think 1.0, seq 32768, LR 8e-5, **1,723 steps = 1.81B tokens = 7.3% of an epoch**, HF exports every 345 steps. There is no 0.5-epoch dense Think run. Its step1723 export is being evaluated under the paper protocol from #1875 as a cheap lower bound on the dense lift curve; results will be posted there.

## Traps already hit on this path

- SFT from the HF base diverges to CE ≈ 8 at every LR unless `--rope_scaling_factor 8` is passed (Olmo 3 YaRN). Verify the loaded config actually carries YaRN before spending GPU-days.
- Context parallelism and rope scaling conflicted in an earlier Olmo 3 SFT; the full script uses `cp_degree 2` with the factor, so check the first 30 steps' CE against a no-CP gate before trusting it.
- holmes allocated jobs are shielded for 8h only, then preempted when the group is over allocation; use `--resume_from_checkpoint` (#1823, on main) and `--ephemeral_save_interval`, and expect a new output dir per launch. #1875 recovered twice this way with identical CE replay.
- `datasets>=4.8.4,<5` or tokenization dies at the last step after ~11h (torchvision VideoReader).
- olmo-eval jobs: gantry installs deps one by one, last writer wins; `transformers==5.14.1,huggingface-hub==1.16.1` must be the last entries or the battery fails at import.
- Under a chat template a base model rambles to the cap; bases evaluated for a lift baseline are floors, not capability numbers.

## Deliverables

1. Runs list (gate, train segments, evals) in the `Runs:` format.
2. One table: retrained SFT vs released Think-SFT vs base, same protocol, with cap-hit rates.
3. A one-line verdict on whether the pipeline reproduces the released model, and the measured GPU-h.