Tencent / Tencent/workbuddy-bench
Register `tencent/workbuddy-bench` as an official Benchmark dataset on the Hugging Face Hub
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 356
- Forks
- 38
- PR merge metrics
- No merged PRs in 30d
Description
Hi team! thanks for open-sourcing the benchmark and the eval harness :)
I noticed the dataset already lives on the Hub at tencent/workbuddy-bench, but it isn't registered as a Hub Benchmark yet. Registering it would upgrade it from "just a dataset" to a benchmark the Hub actively tracks, with a few concrete benefits:
- Dynamic leaderboard: the Hub aggregates evaluation results submitted to any model repo and renders a live, sortable leaderboard on the dataset page, no need to hand-maintain a leaderboard table/site.
- Discoverability: official benchmarks get a "Benchmark" tag and show up in the official benchmarks browser, so people looking for agentic/coding-agent evals can find WorkBuddy Bench without already knowing about it.
- Results travel with the model: once someone evaluates a model and submits a small YAML to that model's repo, the score shows up both on the model page (linking back to your leaderboard) and on your leaderboard.
- Programmatic access: once registered, anyone (including agents/scripts) can pull the leaderboard via
HfApi().get_dataset_leaderboard("tencent/workbuddy-bench")orGET /api/datasets/tencent/workbuddy-bench/leaderboard, and per-model results viamodel_info(..., expand=["evalResults"]). Also embeddable as an iframe on workbuddybench.com if useful. - You can directly add info about what harness was used for each model, allowing you to compare harnesses on the hub.
Docs:
- Registering a benchmark: https://huggingface.co/docs/hub/eval-results#registering-a-benchmark
- Fetching leaderboard data via API: https://huggingface.co/docs/hub/leaderboard-data-guide
The process is pretty lightweight: add an eval.yaml to the dataset repo root describing the benchmark/tasks (see GPQA, SWE-bench Verified, or HLE's eval.yaml for examples), then we will add you to the allowlist of benchmarks!
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in the dataset repository root by reading the Hugging Face documentation on registering benchmarks and comparing the eval.yaml examples for GPQA, SWE-bench Verified, and HLE. Add an eval.yaml describing WorkBuddy Bench's benchmark and tasks, then verify that the dataset is accepted for the official benchmark allowlist.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- huggingface, yaml
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100