Automattic / Automattic/studio
Studio Code: compose prompts and tools from resolved site capabilities
- Dominant language
- TypeScript
- Stars
- 517
- Forks
- 95
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 162
Description
## Problem
Studio Code currently asks the model to discover WordPress.com site plan context inside the agent loop. In remote-site mode, the system prompt instructs the agent to call `GET /` before doing any work, read `plan.product_slug`, and then infer what is allowed from static prompt prose.
That makes the prompt responsible for capability discovery, capability interpretation, tool affordance shaping, and user-facing explanation. It also makes plan/capability drift easy: the model sees broad tools and endpoint guidance even when the active site cannot use those capabilities.
## Current behavior observed
- `apps/cli/ai/system-prompt.ts` tells the agent to call `GET /` with `apiNamespace: ""` before work and check `plan.product_slug`.
- `apps/cli/ai/system-prompt.ts` hardcodes remote plan/design guidance and lists broad WP.com endpoint families, including plugin/theme endpoints.
- `apps/cli/ai/agent.ts` builds remote MCP tools with `createRemoteSiteTools( wpcomAccessToken, activeSite.wpcomSiteId! )`; no site capability object is passed.
- `apps/cli/ai/tools.ts` exposes remote tools via `createRemoteSiteTools( token, siteId )`, which always includes the same generic `wpcom_request` plus site/screenshot/pull tools.
- `apps/cli/ai/wpcom-tools.ts` strips `plan.features` from the `GET /` response and comments that the system prompt hardcodes what each plan tier can do.
- Remote `SiteInfo` / `site.selected` events carry `name`, `url`, and `wpcomSiteId`, but not plan/capability metadata.
- Sync code elsewhere already derives at least one capability dynamically (`studio-sync`) from `site.plan.features.active`, but that normalized information is not threaded into the agent runtime.
## Proposal
Resolve the active site capabilities before starting the model turn, then compose the prompt, tool descriptions, tool schemas, and runtime gates from that same normalized capability object.
Suggested flow:
```text
User sends message
↓
Studio resolves active site/environment
↓
Studio fetches or refreshes site capability snapshot
↓
Studio builds capability-aware MCP tools and schemas
↓
Studio builds a smaller capability-aware system prompt
↓
Agent starts with a scoped operating environment
```
The model should not need to call `GET /` as a first step just to know what it is allowed to do.
## Suggested capability shape
```ts
type WpcomSiteCapabilities = {
planSlug: string;
planName: string;
isFree: boolean;
isExpired: boolean;
canEditContent: boolean;
canUseCustomCss: boolean;
canInstallPlugins: boolean;
canUploadThemes: boolean;
canRunCustomPhp: boolean;
canUseStudioSync: boolean;
};
```
This should be treated as site-specific, not user-global. The same user can manage multiple WordPress.com sites with different plans/capabilities.
## Desired architecture
- Prompt: summarize current site capabilities and policy, not every endpoint/tool.
- Tool descriptions: describe only actions relevant to the active site capabilities.
- Tool schemas: constrain input where possible, e.g. allowed sync options for the current site.
- Runtime gates: enforce the same capabilities even if the model attempts an unsupported operation.
Example for a Premium/Explorer site:
```text
Available:
- Edit posts/pages/media
- Update supported block content
- Use custom CSS
Unavailable:
- Install plugins
- Upload custom themes
- Run custom PHP
- Push plugin/theme code
```
`site_push` should either omit unsupported options from its schema/description or reject them before execution with a precise capability error.
## Why this matters
- Removes brittle “first call GET /” prompt instructions.
- Reduces prompt size and duplicated tool/endpoint lists.
- Prevents unavailable capabilities from being advertised to the model.
- Keeps capability truth in code instead of prose.
- Makes user-facing refusals more precise and product-aligned.
## Notes
This is separate from any tactical prompt copy correction. The goal is to make Studio Code capability-aware before the model loop starts, then expose a scoped prompt and tool surface for the active site.
Contributor guide
Assessment
This issue has not been assessed yet.