stacklok / stacklok/mecatl

Add browser-based automation tools to the core catalog

Open
#751 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Go
Stars
152
Forks
16
Avg merge
14h 48m
Merged PRs (30d)
536

Description

Summary

mecatl has no way to drive a real browser. The web surface today is two tools: WebSearch (discover sources, engine/adapter/search/websearch.go) and the built-in WebFetch (engine/adapter/webfetch, re-exported from internal/adapter/tools). Per ADR 0105, WebFetch is deliberately bounded: one absolute HTTP(S) URL, GET only, no headers/cookies/credentials, HTML parsed via x/net/html without rendering, script execution, or subresource loading. JavaScript-rendered sites are called out as intentionally unsupported.

That leaves a real gap. An agent cannot:

  • Read a page whose content only renders client-side (SPAs, JS-gated docs).
  • Click, fill forms, or complete multi-step / authenticated flows.
  • Wait on selectors/navigation, or execute page JS.
  • Capture screenshots for visual verification.

This is a request to add first-class browser automation to the tool catalog.

Motivation

A growing class of agent tasks need a rendered, interactive browser — QA of web UIs (including our own mecatl-studio surface), scraping client-rendered content, and driving multi-step flows. Bounded WebFetch correctly refuses these; there is no fallback short of an ad-hoc MCP server or shelling out, which sidesteps the catalog's permission model and the FenceUntrusted guarantees.

Proposed capability

A set of browser tools (minimum viable surface):

  • Navigate to a URL
  • Extract rendered (post-JS) DOM / text
  • Click element, fill + submit form
  • Wait for selector / navigation
  • Screenshot (artifact handling TBD)
  • Optionally: evaluate JS in page context

Fit with existing architecture

  • Where it lives. WebFetch/WebSearch are engine reference adapters re-exported through internal/adapter/tools. If browser automation ships in-tree it likely follows the same shape (engine/adapter/browser + re-export), joining the catalog via tools.All() / Register() in internal/adapter/tools/tools.go. But a headless-browser dependency (Playwright/Chromium) is far heavier than x/net/html, so an MCP-server integration (e.g. Playwright MCP behind the existing MCP seam) may be the better first cut — this is the key design decision to settle. See open questions.
  • ReadOnly() dispatch. Read-style ops (navigate, extract, screenshot) are read-parallel; interaction ops (click, fill, submit, evaluate JS) mutate remote state and must report ReadOnly() == false so the agent loop dispatches them serially (gauntlet #4).
  • Untrusted output. Rendered page text is untrusted and prompt-injection-prone — same trust boundary WebFetch/WebSearch handle. It must go through the agent.FenceUntrusted choke point rather than re-implementing framing in a new adapter.
  • Session profiles. Decide inclusion in All() vs NoFS(). A browser tool needs no workspace, so like WebFetch/FetchMcpResource it could ride in the no-fs profile — but its weight and blast radius argue for it being opt-in / off by default.

Safety considerations (must not regress ADR 0105 posture)

WebFetch closes redirect SSRF and DNS-rebinding paths and refuses private/loopback/link-local/metadata targets. A real browser reopens all of that plus more (arbitrary subresource loads, JS execution, cookie/credential handling, downloads). Requirements to work out before shipping:

  • SSRF / egress controls at least as strong as WebFetch — no navigation to private/metadata/loopback ranges; re-validate on in-page navigations and redirects.
  • Sandboxing / isolation for the browser process; bounded resource use (time, memory, page/tab count).
  • Explicit posture on credentials, cookies, and file downloads (default deny).
  • Default-off / permission-gated, consistent with the guardrails model (ADR 0021).
  • Screenshot/artifact storage and size caps; model-visible output bounded like the 25 KB WebFetch cap.

Open questions

  1. Native in-tree adapter vs. MCP server (Playwright MCP)? Weigh dependency weight, cross-platform browser provisioning, and maintenance vs. the tighter integration/guarantees of an in-tree adapter. (Recommend evaluating MCP-first.)
  2. Headless vs. headed; where do Chromium/Playwright binaries come from in deploy images?
  3. Which session profiles include it, and is it default-off?
  4. Screenshot artifact handling over the gRPC + HTTP/SSE API surface.
  5. Do we need a dedicated ADR (as WebFetch got 0105) before implementation?

Scope / not-in-scope

  • In scope: the tool surface above, its safety model, and catalog/permission wiring.
  • Out of scope (for now): full crawler orchestration, video capture, browser-extension automation.

Filed on behalf of @Derek2Tu. Grounded against main: ADR 0105, internal/adapter/tools/tools.go, engine/adapter/webfetch, engine/adapter/search/websearch.go.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with ADR 0105, internal/adapter/tools/tools.go, engine/adapter/webfetch, and engine/adapter/search/websearch.go to understand existing adapters, catalog profiles, and trust boundaries. Resolve the native adapter versus MCP design and the safety, session, and artifact questions before implementation. Done means an agreed browser-tool surface and documented permission and isolation model.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend, security, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.