Reproduce Olmo-3-7B using post-training repo

Use the post-training repository as the framework for supervised fine-tuning (SFT) and direct preference optimization (DPO) on the pretrained base model.

The first validation milestone is to reproduce the results of Olmo-3-7B-Instruct-SFT starting from the Olmo-3-7B-Think-SFT checkpoint.

The second milestone is to apply DPO on top of the reproduced SFT checkpoint and match the performance of Olmo-3-7B-Instruct-DPO.

The objective for both milestones is to achieve a 50% win-rate against the official SFT and DPO checkpoints of Olmo3-7B.

Tasks

• Reproduce Olmo-3-7B-Instruct-SFT from the Olmo-3-7B-Think-SFT checkpoint, evaluate it against the official SFT model using OpenJury, and confirm that it reaches ~50% win rate.

• Train a DPO checkpoint starting from the reproduced SFT checkpoint, evaluate it against the official Olmo-3-7B-Instruct-DPO model using OpenJury, and confirm that it reaches ~50% win rate.

Document the training configuration, evaluation setup, and final results.