TIGER-AI-Lab / TIGER-AI-Lab/ClawBench
publicity: turn the model leaderboard call-to-action into an evidence-based submission guide
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 795
- Forks
- 58
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 18
Description
Opportunity
README invites readers to submit a model, but links to CONTRIBUTING's general contribution table. That table describes adding a config in gitignored models/models.yaml; there is no dedicated leaderboard result issue form or step-by-step evidence handoff. Publicity readers need a concrete route from running the benchmark to submitting a reviewable result.
Deliverable (inside this repository)
- Add docs/model-submissions.md and a dedicated GitHub issue form; link directly from both README calls to action and CONTRIBUTING.
- Distinguish requesting a model evaluation from submitting completed results.
- Specify corpus revision/task IDs, sample count, model and harness revision, browser runtime, judge/rubrics, timeouts/attempts, Stage 1 and both Stage 2 metrics, infrastructure/inconclusive counts, and exact reproduction command.
- Accept a linked evidence bundle/manifest rather than asking contributors to commit models.yaml or API keys; state which artifacts must be sanitized.
- Explain partial results and maintainer review; submitting a config alone is not an accepted leaderboard row.
- Provide a small clearly labeled synthetic submission example and check its required fields.
Success measure: a first-time contributor can submit sufficient evidence using the guide/form without a maintainer first explaining what to upload. Do not promise leaderboard placement or guaranteed turnaround.
Related: #243/#340 for metric reporting, #347 for provenance, #363/#368 for judge-cache identity, #366 for release readiness. This is a contributor-conversion improvement, not a request to publish a website page.
@Perry2004 please review the required evidence and route submissions to the appropriate maintainer.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the existing leaderboard calls to action in README and the contribution table in CONTRIBUTING, then inspect the repository's GitHub issue-form conventions. Add docs/model-submissions.md and the dedicated form, link both from README and CONTRIBUTING, and verify that the synthetic example covers the required evidence and that the form distinguishes requests from completed submissions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github
- Domain
- documentation, tooling
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100