MiniMax-AI / MiniMax-AI/MiniMax-Provider-Verifier

EvalScope integration: one-command evaluation support for MiniMax-Vendor-Verifier

Open Beginner friendly
#37 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
51
Forks
13
Avg merge
48m
Merged PRs (30d)
2

Description

Hi MiniMax-AI/MiniMax-Provider-Verifier maintainers,

EvalScope now supports MiniMax-Vendor-Verifier. This gives users another way to run the benchmark when they want a standardized evaluation workflow around OpenAI-compatible model endpoints, saved predictions, scoring, and report generation.

This may be useful for users who find this repository while setting up MiniMax-Vendor-Verifier and want a quick path to evaluate a hosted or locally served model.

Example:

evalscope eval \
  --model YOUR_MODEL \
  --api-url OPENAI_API_COMPAT_URL \
  --api-key EMPTY_TOKEN \
  --datasets minimax_verifier \
  --limit 10

Docs: https://evalscope.readthedocs.io/en/latest/benchmarks/minimax_verifier.html
EvalScope repo: https://github.com/modelscope/evalscope

We can also open a small PR adding this as a README/Docs note, linked to this issue.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the README or documentation area where an evaluation workflow note would fit, then review the linked EvalScope benchmark documentation and example command. Done means users can find the EvalScope integration, understand the command and its placeholders, and follow the provided links.

Written by the indexing model from the issue text.

Assessment

Domain
documentation
Issue type
Documentation
Difficulty
1/5
Estimated time
Under an hour
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
64/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.