EvalScope integration: one-command evaluation support for olmOCR-Bench
- 主要語言
- Python
- 星號
- 19.5k
- 分支
- 1.6k
- PR 合併指標
- 30 天內沒有已合併 PR
描述
Hi allenai/olmocr maintainers,
EvalScope now supports olmOCR-Bench. This gives users another way to run the benchmark when they want a standardized evaluation workflow around OpenAI-compatible model endpoints, saved predictions, scoring, and report generation.
This may be useful for users who find this repository while setting up olmOCR-Bench and want a quick path to evaluate a hosted or locally served model.
Example:
```bash
evalscope eval \
--model YOUR_MODEL \
--api-url OPENAI_API_COMPAT_URL \
--api-key EMPTY_TOKEN \
--datasets olmocr_bench \
--limit 10
```
Docs:
- olmOCR-Bench: https://evalscope.readthedocs.io/en/latest/benchmarks/olmocr_bench.html
EvalScope repo: https://github.com/modelscope/evalscope
Some benchmarks may require additional assets, Docker, media files, or judge-model configuration for a full run; the EvalScope guide above includes the benchmark-specific setup notes.
If this is useful for your community, we would be happy to discuss a small README/Docs note linking to the EvalScope usage guide.
貢獻指南
研究方向
Start by locating the repository README or benchmark documentation where an external olmOCR-Bench workflow belongs. Add a concise EvalScope usage note linking to the provided guide and showing the example command, then verify that the links and command are readable and accurate.
由索引模型根據 Issue 內容生成。
評估
- 領域
- documentation
- Issue 類型
- 文件
- 難度
- 1/5
- 預估耗時
- 1-3 小時
- 活躍度
- 活躍
- 描述清晰度
- 基本清楚
- 新手友好度
- 72/100