allenai / allenai/OLMo-core

newer transformers library versions break olmo3, fix provided in a PR

Offen
#685 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
Python
Sterne
1.5k
Forks
315
Ø Merge
1 T. 9 Std.
Gemergte PRs (30 T.)
11

Beschreibung

OLMo 3 scaled RoPE handling was [broken by a some sweeping change refactoring RoPE handling](https://github.com/huggingface/transformers/pull/39847). As a result, using a model such as `allenai/Olmo-3-1025-7B` produces incorrect outputs. I tested this by measuring the Olmo3 paper table 3 results and can see that using f.ex. transformers 5.5.3, the table 3 cannot be reproduced. However, I re-measured with my fix and the results seem to be fixed if I use a transformers version that includes my fix.

Would anyone from Ai2 be willing to review my PR that fixes this: https://github.com/huggingface/transformers/pull/45945

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.