Which benchmark version does the leaderboard use?
Aberta
- Linguagem predominante
- Python
- Estrelas
- 470
- Forks
- 72
- Merge médio
- 5d 14h
- PRs com merge (30d)
- 5
Descrição
The `DATA_TYPE_TO_SOURCE_TO_VERSION` in `molmo_spaces_constants.py` currently pins `molmospaces-bench-v1` to `20260210`, but we found a newer version `20260408` is available on R2. The two versions differ in all `benchmark.json` files — `task_description` was rewritten (e.g. `"Pick up some shiny silver kettle"` → `"pick up the kettle."`) and new fields `task_horizon_sec`/`task_type` were added.
Which version is the [leaderboard](https://molmospaces.allen.ai/leaderboard) currently using — `20260210` or `20260408`? And will it be updated to the newer version?
Guia de contribuição
Nenhum guia de contribuição indexado para este repositório
Avaliação
Esta issue ainda não foi avaliada.