Built-in Harper Agent Component

## Proposal / Motivation

Ship an AI agent **inside** the Harper server process (as opposed to the external `harper-agent` CLI). It receives prompts from app developers and operators, then autonomously develops, tests, monitors, and debugs Harper components and applications running on that instance.

- Single LLM-backed operation API instead of a separate CLI on the dev machine.
- Zero-friction access to live server state: in-process operation dispatch (no auth round-trips, no HTTP), raw log files, component file edits, V8 inspector attach against worker threads.
- Autonomous mode (timers + self-resumption) lets the agent iterate on a goal — e.g. "build a recommendations component, then test it, then refine" — without a human in the loop on each turn.
- Coexists with `harper-agent` CLI and Studio chat: those become thin clients to `agent_prompt` rather than parallel engines.

## Relationship to HarperFast/harper#510 / HarperFast/harper#612 / HarperFast/harper#617

This issue depends on and **builds on top of** the model-access stack rather than duplicating it:

- **Loop**: the agent uses `scope.models.generate(input, { toolMode: 'auto', tools, ... })` from HarperFast/harper#510 + HarperFast/harper#612. No bespoke `@openai/agents` integration; no second loop implementation.
- **Tools**: most tools come from the unified MCP tool registry (#615), populated by the **Operations profile** (#617, e.g. `describe_*`, `search`, `restart`, `set_configuration`, `package_component`, `deploy_component`, …) and the **Application profile** (#618, auto-generated per-Resource tools). The built-in agent consults the registry as the configured agent user, getting RBAC-filtered tools for free.
- **Server-only extensions**: a small set of tools is **private to this component** — never registered in HarperFast/harper#615 — because their runtime assumptions only hold on the main thread (see "Operator-only tools" below).

This is **distinct from** application agents built with HarperFast/harper#612 by app developers in Resource code:

| | Built-in agent (#626) | App-developer agent (#612) |
|---|---|---|
| Audience | Operators + app devs of this Harper instance | End users of the app |
| Runtime | Main thread (server-resident) | Worker thread (per-request) |
| Conversations | `system.hdb_agent_session` (server-local, persistent, operator-owned) | `ConversationResource` (#511, app-defined, app data model) |
| Tools | Full registry **+** operator-only extensions | RBAC-filtered registry only |
| Identity | Configured agent user (default super_user) | Caller's identity |

The two share the *loop* (one implementation in HarperFast/harper#612) but not session state, not tool surface, and not auth identity.

## Architecture

**Where it runs.** Built-in component registered via `HARPER_BUILTIN_COMPONENTS`, loaded by `core/components/componentLoader.ts`. Exports `startOnMainThread` (legacy pattern, like `replication/subscriptionManager.ts`). Lives on the main thread alongside the operations API server (`core/server/operationsServer.ts`, default port 9925). Worker threads handle application/REST traffic; the agent stays out of their way and inspects them via CDP.

**Request flow:**

1. `startOnMainThread` registers `agent_prompt` (and friends) via `server.registerOperation`. These handlers run directly on the main thread.
2. `agent_prompt` appends the user message to the named session and kicks the agent loop (or attaches to an in-flight loop for that session). Returns `{ session_id, message_id }` immediately.
3. Internally, the loop is a call to `scope.models.generate(messages, { toolMode: 'auto', tools: composedToolSet, maxToolIterations, ... })` — the HarperFast/harper#612 orchestrator handles tool dispatch, RBAC, audit-log integration, and re-invocation.
4. Clients **poll** `get_agent_session` to see transcript/status updates (SSE streaming can come later).
5. Concurrent sessions interleave on the event loop (turns are mostly `await`s on LLM/HTTP). Per-session serialization prevents two prompts in the same session from racing.

## Component structure

```
agent/
agent.ts // entry: exports startOnMainThread, handleApplication
operations.ts // registerOperation calls
session.ts // CombinedSession backed by hdb_agent_session table
toolset.ts // composes RBAC-filtered registry + operator-only tools per call
tools/
fsTools.ts // scoped read/write/list/grep against componentsRoot + logDir + configDir
inspectorTool.ts // CDP attach/evaluate/breakpoint/logpoint/profile against worker debug ports
scheduleTool.ts // schedule_followup via setTimeout(...).unref()
httpFetchTool.ts // outbound fetch (web research + self-test against own server)
```

**Wire it up** in `bin/harper.js`:

\`\`\`js
process.env.HARPER_BUILTIN_COMPONENTS = ... + ',agent=@/dist/agent/agent.js';
\`\`\`

No new top-level model-loop code, no provider SDK wiring — that all comes from `scope.models` via HarperFast/harper#510.

## Operations to register

All gated on **super_user**, registered on the main thread in `startOnMainThread` (same pattern as `install_usage_license` in `licensing/usageLicensing.ts`):

| Name | Purpose | Returns |
|---|---|---|
| `agent_prompt` | Append a user message and start/resume the loop | `{ session_id, message_id }` |
| `get_agent_session` | Read session transcript + status | items + status |
| `list_agent_sessions` | Enumerate sessions | array |
| `cancel_agent_run` | Abort an in-progress run | ack |
| `approve_agent_action` | Resolve a pending approval | ack |
| `set_agent_config` | Update model / maxTurns / allowed tools at runtime | ack |

## Tool composition

Per `agent_prompt` invocation, the agent's tool set is composed from two sources:

**1. Unified MCP tool registry (#615), RBAC-filtered for the configured agent user.** With HarperFast/harper#617 + HarperFast/harper#618 this includes:

- Operations API tools (#617): `describe_all`, `describe_table`, `search`, `read_audit_log`, `package_component`, `deploy_component`, `drop_component`, `restart`, `set_configuration`, `add_role`, etc. (Subject to the per-operation allow/deny list and per-op annotations like `destructiveHint`.)
- Per-Resource Application tools (#618): auto-generated `get_*`, `search_*`, `create_*`, `update_*`, `delete_*` for each `@export`-ed Resource on the instance.
- Custom Resource methods opted in via static `mcpTools` (#622).
- `read_log` (the operations API), which lives in HarperFast/harper#617's profile. **Note**: this is *not* the low-level log file reader — that's covered by the scoped FS tools below.

The agent gets these for free; no per-tool wiring in `agent/`.

**2. Operator-only tools, passed inline at the `scope.models.generate` call site.** These are *not* registered in HarperFast/harper#615 because their runtime assumptions only hold on the main thread, and exposing them to application agents would conflict with Harper's app abstractions:

| Tool | Why operator-only |
|---|---|
| `inspector_attach`, `inspector_evaluate`, `inspector_set_breakpoint`, `inspector_set_logpoint`, `inspector_profile_cpu` | Safe only from a thread that isn't the one being inspected. Built-in agent on main thread → can debug workers. App agents on worker threads → would deadlock attaching to themselves. |
| `read_file`, `write_file`, `apply_patch`, `list_dir`, `grep_files`, `tail_file` (all scoped to `componentsRoot` + `logDir` + config-file dir) | Harper deliberately abstracts the filesystem from application code (Resources, not files). Apps mutating component source files is out of layer. Also provides low-level log access without needing a second `read_log` tool. |
| `schedule_followup({ delayMs, prompt })` | Worker threads restart on code reload — timers there get lost. Only the long-lived main thread has the right lifecycle for autonomous follow-up. |
| `http_fetch` | Outbound HTTP for web research and self-testing the agent's own deployed components against `localhost:`. Apps that need outbound HTTP should wrap it in a Resource. |

Operator tools are appended to the per-call `tools:` list passed to `scope.models.generate`. Without registration, there's no path by which they appear in `tools/list` for external MCP clients or for application `toolMode: 'auto'` callers.

## Sessions

New system table `system.hdb_agent_session` keyed by `session_id`, holding an ordered array of `AgentInputItem`s plus a `pendingApprovals: ApprovalRequest[]` field used by the approval flow (`approve_agent_action` resolves entries here). Implements a `CombinedSession` interface so the model-access call can hydrate / persist history transparently.

**Intentionally separate from `ConversationResource` (#511)**, which is the app-developer-facing conversation primitive. Different audience, different lifecycle, different schema — the built-in agent's sessions are operator-owned and server-local; `ConversationResource` conversations are app-defined and may be tenant-scoped, multi-user, indexed, etc.

## Configuration

New `agent:` block in `harperdb-config.yaml`, validated against `core/config-root.schema.json`:

\`\`\`yaml
agent:
enabled: true # default false; opt-in to avoid surprise LLM bills
provider: anthropic # optional — falls back to scope.models default
model: claude-opus-4-7 # optional — falls back to scope.models default
maxTurns: 50
maxCostUsd: 5.00 # per-session hard cap; loop aborts with structured error when hit
autoApprove: false # if true, agent runs without approval gates (still gated by allowDestructive)
allowDestructive: false # required to enable destructive ops tools (drop_component, restart, set_configuration, ...)
user: hdb_agent # role/user the agent acts as (default: hdb_agent super_user, created at startup if missing)
componentsScope: ./components
\`\`\`

**Provider/model resolution**: if `agent.provider` / `agent.model` are omitted, fall back to the `scope.models` default. Single-provider users configure once; power users can override per agent.

**API keys**: env vars only, read at startup, never in config. Inherited from the same env vars `scope.models` already uses (`ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, …).

## Permission / auth model

- `agent_prompt` and friends require **super_user** — enforced by the standard operations auth layer (we don't set `bypass_auth`).
- Tool calls execute under the **agent's configured user**. Default: a system `hdb_agent` super_user created at startup if missing. Operators can swap to a restricted role to limit blast radius (e.g. a read-only role for an analytics agent).
- Destructive operator tools (any that mutate config, drop components, or restart) require `allowDestructive: true` **and** still emit an approval request when `autoApprove: false`. `restart` and `set_configuration` require approval even when `autoApprove: true` — a misfire there bricks a node.
- FS tools reject any path resolving outside the scoped roots.

## Self-debugging support

`threadServer.js` opens the V8 inspector on `THREADS_DEBUG_PORT` (main) or `THREADS_DEBUG_STARTINGPORT + workerIndex` (workers) when `threads_debug=true`. The agent's `inspector_*` tools speak CDP over WS to the **worker** ports. Document in setup that operators must set `threads_debug: true` and a sensible starting port before asking the agent to debug.

`inspector_attach` rejects `workerIndex < 0` (the main thread is where the agent itself runs; attaching would deadlock).

## Dependencies

To `harper-pro/package.json`: nothing new for the loop itself — it's all `scope.models` (#510). The only direct deps:

- `ws` (already transitively present, for CDP client to worker debug ports)
- `zod` if we choose Zod-typed tool schemas for the operator-only tools (already transitively present)

No `@openai/agents`, no `ai`, no `@ai-sdk/*` direct in `harper-pro` — those live in `scope.models` resolution.

## Resolved decisions

- **Loop**: `scope.models.generate(..., { toolMode: 'auto' })` via HarperFast/harper#612. No second implementation.
- **Threading**: ops API is main-thread; component registers directly, no bridge.
- **Tool registry vs private tools**: registry (#615 via HarperFast/harper#617/#618) for everything portable; **operator-only tools stay private to this component** because their runtime constraints (main-thread inspector, FS-as-abstraction-layer, timer persistence) don't translate to application agents.
- **Sessions**: `hdb_agent_session` is separate from `ConversationResource` (#511) by design.
- **Provider/key config**: `agent:` block falls back to `scope.models` defaults; keys env-only.
- **Status updates**: clients poll `get_agent_session`; SSE/streaming later.
- **Approvals**: stored on the session; resolved via `approve_agent_action`.
- **Concurrency**: multiple sessions interleave on the event loop; per-session serialization.
- **Destructive ops**: `allowDestructive` flag; `restart` and `set_configuration` always require approval.
- **Inspector**: workers only.

## Open questions

1. **`harper-agent` CLI transition.** With the in-process built-in agent doing the work, the CLI's role compresses to a transport/UI in front of `agent_prompt`. Worth confirming we want to migrate it to that shape rather than continue to maintain two engines.
2. **Per-session `maxCostUsd` enforcement.** HarperFast/harper#612 needs to expose a per-call token/cost budget for the orchestrator to enforce, or the built-in agent enforces it externally by inspecting cumulative `analytics.model_call` rows for the session. Prefer the former; see HarperFast/harper#612 follow-up.
3. **Cost telemetry surfacing.** Should `get_agent_session` include per-turn cost so operators can see budget burn-down without joining to `analytics.model_call` manually?
4. **First-run UX**. Server-side only — no REPL. Docs page with curl examples + Studio panel later.

## Critical files to read / modify

**Modify:**

- `harper-pro/bin/harper.js` — add `agent=@/dist/agent/agent.js` to `HARPER_BUILTIN_COMPONENTS`.
- `harper-pro/core/config-root.schema.json` — add `agent` block schema.
- `harper-pro/core/utility/hdbTerms.ts` — add config param constants + new `OPERATIONS_ENUM` entries for `agent_prompt`, `get_agent_session`, etc.

**Create:** `harper-pro/agent/` (full new directory, far thinner than the original proposal).

**Read & reuse:**

- `scope.models` model-access API (#510) — backend.
- `scope.models` `toolMode: 'auto'` orchestrator (#612) — loop.
- Unified MCP tool registry (#615) — tool discovery.
- Operations MCP profile (#617) — ops-as-tools.
- Application MCP profile (#618) — Resources-as-tools.
- `harper-pro/licensing/usageLicensing.ts` — exemplar for component shape (`handleApplication`, `registerOperation`, system table reads).
- `harper-agent/tools/files/*` — reusable scoped FS tool implementations to lift.

## Verification

1. **Build & boot**: `npm run build` in harper-pro, start a server with the new env var; check logs for "Agent component initialized". No-op if `agent.enabled=false`.
2. **Round-trip prompt**: `curl -X POST /agent_prompt -d '{"message":"describe the system schema"}'` → returns `session_id` → `curl /get_agent_session/` shows the agent called `describe_all` (from HarperFast/harper#617) and produced a response.
3. **Autonomous build**: prompt "create a Resource called Hello that returns 'hi'" → confirm a component file appears (via scoped FS tools), then `curl localhost:9926/Hello` to confirm the deployed app works.
4. **Self-test loop**: "build X, then call it on localhost:, debug any failures, repeat until working" — verifies `http_fetch` + tool chaining through HarperFast/harper#612.
5. **Debugging**: with `threads_debug=true`, prompt "find why the Foo resource throws on POST" — should attach to inspector, set a logpoint, observe a request, report root cause.
6. **Scheduled work**: "every 5 minutes for the next hour, check cluster status and alert me if any node is down" — verifies `schedule_followup` + persistence across restarts.
7. **Boundary checks**: confirm `read_file('/etc/passwd')` is rejected; confirm `inspector_*` tools are NOT visible to `tools/list` over external MCP; confirm non-super_user callers of `agent_prompt` get 403.
8. **Integration test** at `harper-pro/integrationTests/agent/` with a stub `scope.models` provider so CI doesn't burn LLM credits.