millionco / millionco/reactbench

Please attach which harness was used with every published entry in the data

Open
#2 2 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
271
Forks
11
PR merge metrics
No merged PRs in 30d

Description

I've been playing with Pi a lot recently with multiple agent configurations (which is just to say, presets with model + reasoning effort + subagents all baked in, much like what Amp does). Part of this also means naturally customising the prompts and tools for each agent to maximise each model's performance - and although benchmarks are always not super representative of real-world use, it's super helpful to go look at the harness that does the best job with each model. I was super excited to see ReactBench since I've been writing a lot of React with side-projects recently, and it'd be super helpful to understand which harness eventually gave the scores that you have published. NextJS evals and Artificial Analysis do this too.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the data source and publishing path for ReactBench's published entries; the issue does not name specific files or tests. Trace how each entry is represented, then add the harness used for every published result and verify that the published data displays it consistently.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, react
Domain
documentation, testing-qa
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
62/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.