josharsh / josharsh/webmcp-tools

Benchmark: tool calls vs DOM actuation on the example app

Open
#12 0 comments 0 reactions 0 assignees View on GitHub
dx help wanted
Dominant language
TypeScript
Stars
0
Forks
1
PR merge metrics
No merged PRs in 30d

Description

The WebMCP pitch rests on reliability numbers (fewer errors vs visual scraping). Let's measure it ourselves, honestly, on `examples/todo`:

- Script an agent loop doing N tasks via (a) `executeTool` and (b) a DOM-driving baseline (playwright + a small LLM, or scripted selectors-with-noise)
- Track success rate + latency + tokens
- Publish methodology + results in `docs/benchmark.md` — including where tool calls do *worse*

A credible, self-critical benchmark is worth more than a marketing claim.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.