Tencent / Tencent/workbuddy-bench

Register `tencent/workbuddy-bench` as an official Benchmark dataset on the Hugging Face Hub

Open
#7 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
356
Forks
38
PR merge metrics
No merged PRs in 30d

Description

Hi team! thanks for open-sourcing the benchmark and the eval harness :)

I noticed the dataset already lives on the Hub at tencent/workbuddy-bench, but it isn't registered as a Hub Benchmark yet. Registering it would upgrade it from "just a dataset" to a benchmark the Hub actively tracks, with a few concrete benefits:

  • Dynamic leaderboard: the Hub aggregates evaluation results submitted to any model repo and renders a live, sortable leaderboard on the dataset page, no need to hand-maintain a leaderboard table/site.
  • Discoverability: official benchmarks get a "Benchmark" tag and show up in the official benchmarks browser, so people looking for agentic/coding-agent evals can find WorkBuddy Bench without already knowing about it.
  • Results travel with the model: once someone evaluates a model and submits a small YAML to that model's repo, the score shows up both on the model page (linking back to your leaderboard) and on your leaderboard.
  • Programmatic access: once registered, anyone (including agents/scripts) can pull the leaderboard via HfApi().get_dataset_leaderboard("tencent/workbuddy-bench") or GET /api/datasets/tencent/workbuddy-bench/leaderboard, and per-model results via model_info(..., expand=["evalResults"]). Also embeddable as an iframe on workbuddybench.com if useful.
  • You can directly add info about what harness was used for each model, allowing you to compare harnesses on the hub.

Docs:

The process is pretty lightweight: add an eval.yaml to the dataset repo root describing the benchmark/tasks (see GPQA, SWE-bench Verified, or HLE's eval.yaml for examples), then we will add you to the allowlist of benchmarks!

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in the dataset repository root by reading the Hugging Face documentation on registering benchmarks and comparing the eval.yaml examples for GPQA, SWE-bench Verified, and HLE. Add an eval.yaml describing WorkBuddy Bench's benchmark and tasks, then verify that the dataset is accepted for the official benchmark allowlist.

Written by the indexing model from the issue text.

Assessment

Tech stack
huggingface, yaml
Domain
machine-learning
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.