allenai / allenai/olmocr

EvalScope integration: one-command evaluation support for olmOCR-Bench

未關閉 適合新手
#478 0 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
主要語言
Python
星號
19.5k
分支
1.6k
PR 合併指標
30 天內沒有已合併 PR

描述

Hi allenai/olmocr maintainers,

EvalScope now supports olmOCR-Bench. This gives users another way to run the benchmark when they want a standardized evaluation workflow around OpenAI-compatible model endpoints, saved predictions, scoring, and report generation.

This may be useful for users who find this repository while setting up olmOCR-Bench and want a quick path to evaluate a hosted or locally served model.

Example:

```bash
evalscope eval \
--model YOUR_MODEL \
--api-url OPENAI_API_COMPAT_URL \
--api-key EMPTY_TOKEN \
--datasets olmocr_bench \
--limit 10
```

Docs:
- olmOCR-Bench: https://evalscope.readthedocs.io/en/latest/benchmarks/olmocr_bench.html

EvalScope repo: https://github.com/modelscope/evalscope

Some benchmarks may require additional assets, Docker, media files, or judge-model configuration for a full run; the EvalScope guide above includes the benchmark-specific setup notes.

If this is useful for your community, we would be happy to discuss a small README/Docs note linking to the EvalScope usage guide.

貢獻指南

開啟貢獻指南

研究方向

Start by locating the repository README or benchmark documentation where an external olmOCR-Bench workflow belongs. Add a concise EvalScope usage note linking to the provided guide and showing the example command, then verify that the links and command are readable and accurate.

由索引模型根據 Issue 內容生成。

評估

領域
documentation
Issue 類型
文件
難度
1/5
預估耗時
1-3 小時
活躍度
活躍
描述清晰度
基本清楚
新手友好度
72/100

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。