ADR: agent-browser Integration for Web Automation Skills and Agents

# ADR-027: agent-browser Integration for Web Automation

## Status
**Proposed** | Priority: High | Complexity: Medium

## Context and Problem Statement

Claude Flow V3 currently lacks native browser automation capabilities. Users need to manually orchestrate web interactions, scraping, and testing workflows. The [agent-browser](https://github.com/vercel-labs/agent-browser) package from Vercel Labs provides a production-ready, AI-optimized browser automation CLI that could significantly enhance claude-flow's capabilities.

**Current Gap:**
- No built-in web automation agents
- No browser interaction skills
- Manual coordination for web scraping/testing tasks
- High token usage when agents describe DOM interactions

**Opportunity:**
- agent-browser reduces context usage by **93%** via snapshot refs
- Native Rust CLI provides **instant command parsing**
- 50+ commands cover all browser automation needs
- Session isolation enables multi-agent browser coordination

## Decision Drivers

1. **AI-First Design**: agent-browser's snapshot system returns accessibility tree with refs (`@e1`, `@e2`) instead of full DOM, optimized for LLM token efficiency
2. **Universal Compatibility**: Works with Claude Code, Cursor, Codex, Copilot, Gemini, and more
3. **Production Ready**: Apache-2.0 license, active development (v0.6.0), Vercel Labs backing
4. **Minimal Dependencies**: Only playwright-core, ws, and zod
5. **Session Isolation**: Each session maintains independent browser state (critical for swarm coordination)

## Considered Options

### Option 1: Full Integration (Recommended)
- New `@claude-flow/browser` package
- `browser-agent` agent type
- `/browser` skill with all 50+ commands
- MCP tools for browser operations
- Memory integration for session persistence

### Option 2: Skill-Only Integration
- `/browser` skill wrapping CLI commands
- No dedicated agent type
- Lighter weight, faster to implement

### Option 3: External Dependency Only
- Document agent-browser as recommended companion
- No native integration
- Users manually orchestrate

## Decision Outcome

**Chosen: Option 1 - Full Integration**

Rationale: The 93% context reduction and AI-first design align perfectly with claude-flow's mission. Full integration enables:
- Swarm-based web scraping with coordinated agents
- Automated testing workflows
- Web research agents with persistent sessions
- Screenshot-driven debugging

---

## Domain-Driven Design (DDD)

### Bounded Context: Browser Automation

```
┌─────────────────────────────────────────────────────────────────┐
│ Browser Automation Context │
├─────────────────────────────────────────────────────────────────┤
│ Aggregates: │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ Session │ │ Page │ │ Network │ │
│ │ Aggregate │ │ Aggregate │ │ Aggregate │ │
│ └─────────────┘ └─────────────┘ └─────────────┘ │
│ │
│ Value Objects: │
│ - ElementRef (@e1, @e2) - Viewport │
│ - Snapshot - Cookie │
│ - Selector - StorageItem │
│ │
│ Domain Events: │
│ - PageNavigated - ElementClicked │
│ - SnapshotTaken - FormFilled │
│ - SessionCreated - NetworkIntercepted │
└─────────────────────────────────────────────────────────────────┘
```

### Integration with Existing Contexts

```
┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐
│ Swarm Context │────▶│ Browser Context │◀────│ Memory Context │
│ │ │ │ │ │
│ - Coordinator │ │ - BrowserAgent │ │ - SessionState │
│ - Task Routing │ │ - PageSnapshot │ │ - CookieCache │
└──────────────────┘ └──────────────────┘ └──────────────────┘
│ │ │
▼ ▼ ▼
┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐
│ Hooks Context │ │ MCP Context │ │ Skills Context │
│ │ │ │ │ │
│ - pre-browse │ │ - browser/* │ │ - /browser │
│ - post-browse │ │ - snapshot/* │ │ - /scrape │
└──────────────────┘ └──────────────────┘ └──────────────────┘
```

---

## Technical Architecture

### 1. Package Structure

```
v3/@claude-flow/browser/
├── src/
│ ├── domain/
│ │ ├── session.ts # Session aggregate
│ │ ├── page.ts # Page aggregate
│ │ ├── network.ts # Network aggregate
│ │ └── value-objects/
│ │ ├── element-ref.ts # @e1, @e2 refs
│ │ ├── snapshot.ts # Accessibility tree
│ │ └── selector.ts # CSS/ARIA selectors
│ ├── application/
│ │ ├── browser-service.ts # Core service
│ │ ├── snapshot-service.ts # Snapshot management
│ │ └── session-manager.ts # Multi-session
│ ├── infrastructure/
│ │ ├── agent-browser-adapter.ts # CLI wrapper
│ │ ├── memory-persistence.ts # State storage
│ │ └── mcp-tools.ts # MCP integration
│ └── index.ts
├── package.json
└── README.md
```

### 2. Agent Type Definition

```yaml
# agents/browser-agent.yaml
name: browser-agent
description: Web automation specialist using agent-browser
capabilities:
- web-navigation
- form-interaction
- screenshot-capture
- network-interception
- session-management
routing:
complexity: medium
model: sonnet # Visual/DOM reasoning
tools:
- browser/open
- browser/snapshot
- browser/click
- browser/fill
- browser/screenshot
memory:
namespace: browser-sessions
persist: true
```

### 3. Skill Definition

```yaml
# skills/browser.yaml
name: browser
description: Web browser automation with AI-optimized snapshots
commands:
- name: open
description: Navigate to URL
usage: /browser open
- name: snapshot
description: Get accessibility tree with element refs
usage: /browser snapshot [-i interactive] [-c compact]
- name: click
description: Click element by ref or selector
usage: /browser click <@ref|selector>
- name: fill
description: Fill form input
usage: /browser fill <@ref|selector> "text"
- name: screenshot
description: Capture screenshot
usage: /browser screenshot [path] [--full]
- name: eval
description: Execute JavaScript
usage: /browser eval "code"
```

### 4. MCP Tools

```typescript
// New MCP tools for browser automation
const browserTools = [
{
name: 'browser/open',
description: 'Navigate to URL',
inputSchema: {
type: 'object',
properties: {
url: { type: 'string', description: 'URL to navigate to' },
session: { type: 'string', description: 'Session name (optional)' },
waitUntil: { enum: ['load', 'domcontentloaded', 'networkidle'] }
},
required: ['url']
}
},
{
name: 'browser/snapshot',
description: 'Get accessibility tree with element refs for AI interaction',
inputSchema: {
type: 'object',
properties: {
interactive: { type: 'boolean', default: true },
compact: { type: 'boolean', default: true },
depth: { type: 'number' },
selector: { type: 'string' }
}
}
},
{
name: 'browser/click',
description: 'Click element using ref (@e1) or selector',
inputSchema: {
type: 'object',
properties: {
target: { type: 'string', description: 'Element ref (@e1) or CSS selector' }
},
required: ['target']
}
},
{
name: 'browser/fill',
description: 'Fill form input',
inputSchema: {
type: 'object',
properties: {
target: { type: 'string' },
value: { type: 'string' }
},
required: ['target', 'value']
}
},
{
name: 'browser/screenshot',
description: 'Capture screenshot (returns base64 or saves to path)',
inputSchema: {
type: 'object',
properties: {
path: { type: 'string' },
fullPage: { type: 'boolean', default: false }
}
}
},
// ... 45+ more tools mapping to agent-browser commands
];
```

### 5. Swarm Integration

```typescript
// Browser swarm workflow example
const browserSwarm = {
topology: 'hierarchical',
agents: [
{ type: 'browser-agent', role: 'navigator', session: 'nav-1' },
{ type: 'browser-agent', role: 'scraper', session: 'scrape-1' },
{ type: 'browser-agent', role: 'validator', session: 'test-1' },
{ type: 'researcher', role: 'analyzer' }
],
workflow: [
{ agent: 'navigator', task: 'open and authenticate' },
{ agent: 'scraper', task: 'extract data using snapshots' },
{ agent: 'validator', task: 'verify extracted data' },
{ agent: 'analyzer', task: 'synthesize findings' }
]
};
```

### 6. Memory Integration

```typescript
// Persist browser session state
interface BrowserSessionState {
sessionId: string;
cookies: Cookie[];
localStorage: Record;
currentUrl: string;
lastSnapshot: Snapshot;
history: string[];
}

// Store in claude-flow memory
await memoryStore({
namespace: 'browser-sessions',
key: `session:${sessionId}`,
value: sessionState,
ttl: 3600 // 1 hour
});
```

---

## Implementation Plan

### Phase 1: Core Integration (Week 1-2)
- [ ] Create `@claude-flow/browser` package
- [ ] Implement agent-browser CLI adapter
- [ ] Add `browser-agent` type to agent registry
- [ ] Basic MCP tools (open, snapshot, click, fill)

### Phase 2: Skill & Hooks (Week 3)
- [ ] Create `/browser` skill
- [ ] Add `pre-browse` and `post-browse` hooks
- [ ] Implement session persistence in memory

### Phase 3: Swarm Coordination (Week 4)
- [ ] Multi-session browser swarm support
- [ ] Parallel scraping workflows
- [ ] Screenshot-based debugging integration

### Phase 4: Advanced Features (Week 5+)
- [ ] Network interception tools
- [ ] Auth state management
- [ ] Trace recording for debugging
- [ ] Integration tests

---

## Dependencies

```json
{
"dependencies": {
"agent-browser": "^0.6.0"
},
"peerDependencies": {
"@claude-flow/cli": "^3.0.0-alpha.140"
}
}
```

## Risks and Mitigations

| Risk | Likelihood | Impact | Mitigation |
|------|------------|--------|------------|
| Playwright dependency size | Medium | Low | Use playwright-core (smaller) |
| Browser binary management | Medium | Medium | Leverage agent-browser install |
| Session state corruption | Low | Medium | Memory checkpoints + recovery |
| Rate limiting by sites | High | Medium | Built-in delays + session rotation |

## Success Metrics

- [ ] 90%+ context reduction vs raw DOM approaches
- [ ] <100ms command latency for cached sessions
- [ ] Support for 10+ concurrent browser sessions
- [ ] Zero configuration for basic use cases

---

## References

- [agent-browser npm](https://www.npmjs.com/package/agent-browser)
- [agent-browser GitHub](https://github.com/vercel-labs/agent-browser)
- [agent-browser Website](https://agent-browser.dev/)
- [Playwright Documentation](https://playwright.dev/)

---

## Labels
`enhancement`, `adr`, `ddd`, `integration`, `v3`

Start by reviewing the proposed v3/@claude-flow/browser package structure, especially the TypeScript services and agent-browser adapter, then compare the agents/browser-agent.yaml and skills/browser.yaml entry points with the MCP tool definitions. Done means the planned browser package, agent type, skill, session persistence, swarm support, and integration tests are implemented across the listed phases and meet the stated success metrics.

Written by the indexing model from the issue text.