allenai / allenai/olmocr

EvalScope integration: one-command evaluation support for olmOCR-Bench

Offen
#478 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
Python
Sterne
19.5k
Forks
1.6k
PR-Merge-Kennzahlen
Keine gemergten PRs in 30 T.

Beschreibung

Hi allenai/olmocr maintainers,

EvalScope now supports olmOCR-Bench. This gives users another way to run the benchmark when they want a standardized evaluation workflow around OpenAI-compatible model endpoints, saved predictions, scoring, and report generation.

This may be useful for users who find this repository while setting up olmOCR-Bench and want a quick path to evaluate a hosted or locally served model.

Example:

```bash
evalscope eval \
--model YOUR_MODEL \
--api-url OPENAI_API_COMPAT_URL \
--api-key EMPTY_TOKEN \
--datasets olmocr_bench \
--limit 10
```

Docs:
- olmOCR-Bench: https://evalscope.readthedocs.io/en/latest/benchmarks/olmocr_bench.html

EvalScope repo: https://github.com/modelscope/evalscope

Some benchmarks may require additional assets, Docker, media files, or judge-model configuration for a full run; the EvalScope guide above includes the benchmark-specific setup notes.

If this is useful for your community, we would be happy to discuss a small README/Docs note linking to the EvalScope usage guide.

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.