Azure / Azure/azure-dev

Add connections, models, tools, and skills to Foundry Agent projects after init

Open
#8,049 4 comments 0 reactions 1 assignee Claimed by @therealjohn View on GitHub
ext-agents feature
Dominant language
Go
Stars
569
Forks
364
Avg merge
2d 19h
Merged PRs (30d)
136

Description

We need `azd ai project add` commands that let developers grow their Foundry agent projects after init. Today, if you didn't pick the right tools and connections up front, you're left reverse-engineering samples and hand-editing YAML until something works.

**Scope:** `azd ai project add` commands for tools, connections, model deployments, and skills. Config-only -- the CLI updates `azure.yaml` and tells you what code or infra changes to make. No code generation, no Bicep generation. Remove is done with AZD down.

**Depends on:** [Unify Foundry agent configuration in azure.yaml](https://github.com/Azure/azure-dev/issues/7962). The `add` commands modify the `azure.ai.project` service's `config` block, which is the single source of truth for project-scoped Foundry resources. That restructuring must land first.

---

## Current Problems

1. **Init is a one-shot deal.** You pick everything at `azd ai agent init` time and that's it. No CLI command to add a tool or connection afterward. `npm init` has `npm install`. `dotnet new` has `dotnet add`. We have nothing.

2. **The workaround is "go find a sample."** When someone needs to bolt on Azure AI Search or an MCP tool post-init, they dig through docs, find a sample that happens to use that component, extract the relevant bits, hand-edit YAML, and hope the wiring works. This is where most people give up.

3. **Three files, two templating syntaxes, four layers deep.** Adding a single MCP tool today means touching `agent.yaml` (tool definition), `agent.manifest.yaml` (parameter declaration with `{{handlebars}}`), azure.yaml config (connection with `${ENV}` refs), and possibly Bicep. Nobody can answer "where do I make this change?" without knowing the extension's internals. The init code that processes manifest resources spans over 200 lines across `extractToolboxAndConnectionConfigs` and `extractConnectionConfigs` in [`init.go`](https://github.com/Azure/azure-dev/blob/main/cli/azd/extensions/azure.ai.agents/internal/cmd/init.go) -- 200 lines of wiring that developers are expected to replicate by hand.

4. **Zero discoverability.** There's no help listing available capabilities. Developers have no way to find out what they can add, what each capability requires, or what the config should look like. The extension's registered commands ([`root.go`](https://github.com/Azure/azure-dev/blob/main/cli/azd/extensions/azure.ai.agents/internal/cmd/root.go)) include init, run, invoke, show, monitor, session, files, mcp -- nothing for composition after init.

---

## Solution

The `add` commands live in a new `azd ai project` extension, matching the `azure.ai.project` service kind from [#7962](https://github.com/Azure/azure-dev/issues/7962). This extension owns toolboxes, connections, model deployments, and skills. The existing `azd ai agent` extension keeps doing what it does now -- init, run, invoke, deploy. The split matches azure.yaml: `host: azure.ai.project` for shared resources, `host: azure.ai.agent` for agent runtimes.

A set of `azd ai project add ` commands. Each one:

- **Touches `azure.yaml` only.** Specifically the `azure.ai.project` service's `config` block (toolboxes, connections, deployments) as defined in [#7962](https://github.com/Azure/azure-dev/issues/7962). The CLI becomes the primary way to change agent config. Hand-editing still works, but you shouldn't need it.
- **Tells you what else to do.** After updating config, the command prints whatever code or infra changes remain. Those belong to coding agents or the developer -- not us.
- **Works interactively and headlessly.** Prompts by default, `--no-prompt` for CI/CD and coding agents. Every prompt has a flag equivalent.

The command taxonomy:

| Command | What it adds to `azure.ai.project` config |
|---|---|
| `azd ai project add connection ` | A project connection (category, target, authType, credentials). Supported categories: CustomKeys, AzureOpenAI, CognitiveSearch, ApiKey, RemoteTool, and [20+ more](https://github.com/Azure/azure-dev/blob/main/cli/azd/extensions/azure.ai.agents/internal/pkg/agents/agent_yaml/yaml.go). |
| `azd ai project add model ` | A model deployment (format, name, version, SKU) |
| `azd ai project add skill ` | A skill definition (filesystem-based, reusable capability -- new azure.yaml schema primitive). **Gated on schema design; ships only after the schema is ratified.** |

---

## Detailed Design

### Command Structure

Every `add` command follows the same shape:

```
azd ai project add [name-or-subtype] [flags]
```

Flags shared across all commands:

| Flag | Description |
|---|---|
| `--no-prompt` | Non-interactive mode. All required values must be provided via flags. |
| `--output json` | Emit structured JSON instead of human-readable output. |
| `--project-file` | Path to azure.yaml. Defaults to `./azure.yaml`. |

### `azd ai project add connection`

```
azd ai project add connection [flags]
```

`` is the connection name -- it maps to the `name` field on the [`Connection` struct](https://github.com/Azure/azure-dev/blob/main/cli/azd/extensions/azure.ai.agents/internal/project/config.go) and becomes the key in azure.yaml's `connections:` list. This is also the name used in the Foundry project API when the connection is created at deploy time.

Adds a connection to the `azure.ai.project` config block. A connection is any external resource the project talks to -- an MCP server, an Azure OpenAI endpoint, a search index, a custom API.

The connection properties map directly to the existing [`Connection` struct](https://github.com/Azure/azure-dev/blob/main/cli/azd/extensions/azure.ai.agents/internal/project/config.go) and the [`ConnectionResource` in the manifest schema](https://github.com/Azure/azure-dev/blob/main/cli/azd/extensions/azure.ai.agents/internal/pkg/agents/agent_yaml/yaml.go):

| Flag | Property | Description |
|---|---|---|
| `--category` | `category` | Connection category. Required. See supported categories below. |
| `--target` | `target` | Target endpoint URL or ARM resource ID. Required. |
| `--auth-type` | `authType` | Authentication type. Defaults vary by category. |
| `--api-key` | `credentials.key` | API key value (for ApiKey auth). Stored in azd env, not azure.yaml. Supports `${ENV_VAR}` syntax to read from an existing azd environment variable (e.g., `--api-key '${AZURE_OPENAI_KEY}'`). |
| `--keys` | `credentials` | Key-value pairs for CustomKeys auth (e.g., `--keys x-api-key=val`). Values support `${ENV_VAR}` syntax. Stored in azd env. |
| `--metadata` | `metadata` | Optional metadata key-value pairs (e.g., `--metadata type=custom_MCP`). |
| `--connection-id` | -- | ARM resource ID of an existing Foundry project connection (e.g., `/subscriptions/.../connections/my-conn`). When set, the connection is treated as Foundry-managed -- credential values use `${{connections..credentials.}}` syntax and Foundry resolves them server-side. |
| `--env-var` | `environment` | Map a Foundry-managed secret to an agent environment variable (e.g., `--env-var GITHUB_MCP_TOKEN=${{connections.github-mcp-conn.credentials.x-api-key}}`). Used with `--connection-id`. |

**Supported connection categories** (from the extension's `CategoryKind` enum):

| Category | Typical use |
|---|---|
| `RemoteTool` | MCP servers, A2A agents, external tool endpoints |
| `CustomKeys` | Any service with custom credential fields |
| `ApiKey` | Simple API key auth |
| `AzureOpenAI` | Azure OpenAI endpoints |
| `CognitiveSearch` | Azure AI Search |
| `CognitiveService` | Azure Cognitive Services |
| `Serverless` | Serverless model endpoints |
| `BingLLMSearch` | Bing grounding |
| Others | AzureBlob, Redis, S3, Snowflake, AzureSqlDb, AzurePostgresDb, ADLSGen2, Elasticsearch, Pinecone, Qdrant, etc. |

**Supported auth types** (from the extension's `AuthType` enum):

`AAD`, `ApiKey`, `CustomKeys`, `None`, `OAuth2`, `PAT`, `UserEntraToken`, `AgenticIdentity`, `ProjectManagedIdentity`, `ServicePrincipal`, `UsernamePassword`, `AccessKey`, `AccountKey`, `SAS`

**Example -- adding an MCP tool connection (interactive):**

```
$ azd ai project add connection github-mcp-conn

? Connection category: RemoteTool
? Target URL: https://api.githubcopilot.com/mcp
? Auth type: CustomKeys
? Credential key name: x-api-key
? Credential value: ********
? Metadata (key=value, blank to skip): type=custom_MCP

[x] Added connection 'github-mcp-conn' to project config.
Updated: azure.yaml
Credential stored in azd environment (not in azure.yaml).

Next steps:
- Reference this connection in a toolbox tool entry.
- Run 'azd deploy' to push the updated config to Foundry.
```

**Example -- adding an Azure OpenAI connection:**

```
$ azd ai project add connection my-aoai \
--category AzureOpenAI \
--target https://my-instance.openai.azure.com \
--auth-type ApiKey \
--api-key '${AZURE_OPENAI_KEY}'

[x] Added connection 'my-aoai' to project config.
Updated: azure.yaml
Credential stored in azd environment (not in azure.yaml).
```

**Example -- non-interactive mode (coding agent friendly):**

```
$ azd ai project add connection tavily-mcp \
--category RemoteTool \
--target https://mcp.tavily.com/mcp \
--auth-type CustomKeys \
--keys x-api-key=${TAVILY_API_KEY} \
--metadata type=custom_MCP \
--no-prompt --output json
{
"status": "added",
"type": "connection",
"connection": {
"name": "tavily-mcp",
"category": "RemoteTool",
"target": "https://mcp.tavily.com/mcp",
"authType": "CustomKeys"
},
"files_modified": ["azure.yaml"],
"next_steps": [
"Reference this connection in a toolbox tool entry.",
"Run 'azd deploy' to push the updated config to Foundry."
]
}
```

**Example -- MCP tool connection with Foundry-managed secrets (no secret on disk):**

This scenario avoids storing any secret on disk. The developer creates the connection directly in Foundry using the `az` CLI, then adds it to the azd project config using `azd ai project add connection`.

Step 1: Create the connection in Foundry (secret goes straight to Foundry, never stored locally):

```
$ az cognitiveservices account project connection create \
--name github-mcp-conn \
--resource-group my-rg \
--account-name my-foundry-account \
--project-name my-project \
--connection-category RemoteTool \
--target https://api.githubcopilot.com/mcp \
--auth-type CustomKeys \
--credentials key1=x-api-key val1=ghp_xxxxxxxxxxxx
```

See [`az cognitiveservices account project connection create`](https://learn.microsoft.com/en-us/cli/azure/cognitiveservices/account/project/connection?view=azure-cli-latest#az-cognitiveservices-account-project-connection-create) for full parameter reference.

> **Note:** The `connection create` command in `az` may need changes to fully support this scenario -- in particular, the `--credentials` format for CustomKeys auth and the RemoteTool category may not be supported yet.

Step 2: Add the connection to the azd project config. The command detects connections that already exist in the Foundry project and lets the developer select one interactively:

```
$ azd ai project add connection github-mcp-conn

? Connection category: RemoteTool
? Select an existing Foundry connection or create new:
> github-mcp-conn (existing - RemoteTool)
Create new connection
? Environment variable name for the secret: GITHUB_MCP_TOKEN
? Secret value: Use Foundry reference (${{connections.github-mcp-conn.credentials.x-api-key}})

[x] Added connection 'github-mcp-conn' to project config.
Updated: azure.yaml
Secret is Foundry-managed -- not stored on disk.
Environment variable GITHUB_MCP_TOKEN will be injected at runtime.

Next steps:
- Reference this connection in a toolbox tool entry.
- Run 'azd deploy' to push the updated config to Foundry.
```

The resulting azure.yaml config uses `${{}}` syntax for the credential. The Foundry backend resolves this server-side and injects the secret value into the agent's environment variable at runtime:

```yaml
connections:
github-mcp-conn:
category: RemoteTool
target: https://api.githubcopilot.com/mcp
authType: CustomKeys
credentials:
x-api-key: ${{connections.github-mcp-conn.credentials.x-api-key}}

# In the agent service config, the secret is mapped to an env var:
services:
my-agent:
host: azure.ai.agent
config:
environment:
GITHUB_MCP_TOKEN: ${{connections.github-mcp-conn.credentials.x-api-key}}
```

The developer never sees or stores the secret. The `${{}}` reference is the only thing written to azure.yaml, and the Foundry backend handles injection at deploy/runtime.

**Example -- Foundry-managed connection, non-interactive (`--no-prompt`):**

```
$ azd ai project add connection github-mcp-conn \
--category RemoteTool \
--target https://api.githubcopilot.com/mcp \
--auth-type CustomKeys \
--connection-id /subscriptions/.../projects/my-project/connections/github-mcp-conn \
--keys x-api-key=${{connections.github-mcp-conn.credentials.x-api-key}} \
--env-var GITHUB_MCP_TOKEN=${{connections.github-mcp-conn.credentials.x-api-key}} \
--no-prompt

[x] Added connection 'github-mcp-conn' to project config.
Updated: azure.yaml
Secret is Foundry-managed -- not stored on disk.
Environment variable GITHUB_MCP_TOKEN will be injected at runtime.
```

**Example -- idempotent behavior:**

```
$ azd ai project add connection github-mcp-conn
Connection 'github-mcp-conn' already exists in project config. No changes made.
```

#### Two Secrets Models

Connections can manage credentials in two ways:

**1. azd-environment-managed (what `azd ai project add connection` does):**
- Secret is provided interactively (or via `--api-key '${ENV_VAR}'` syntax in `--no-prompt` mode).
- Secret is stored in `.azure/{env}/.env` -- lives on disk, scoped to the azd environment.
- azure.yaml references it with `${ENV_VAR}` syntax (e.g., `${MY_CONN_API_KEY}`).
- At deploy time, azd resolves the variable and sends the value to Foundry.
- One tool, no external setup.

**2. Foundry-managed (pre-existing connection, reference-only):**
- The connection is created externally -- via `az` CLI, Azure Portal, or the Foundry Toolkit.
- The secret lives server-side in Foundry. Never touches the developer's disk.
- azure.yaml references credential values with Foundry's lookup syntax: `${{connections..credentials.}}`.
- At deploy time, Foundry resolves the reference server-side.
- The enterprise path: secrets managed by an admin, developers just reference them.

`azd ai project add connection` only supports mode 1. Mode 2 doesn't need a CLI command -- the connection already exists. When adding a connection-backed tool to a toolbox (hand-edit or future toolbox command), you can reference Foundry-managed connections using the `${{}}` syntax directly:

```yaml
# In azure.ai.project config, referencing a Foundry-managed connection
toolboxes:
default:
tools:
- type: mcp
connection: ${{connections.github-mcp-conn.credentials.key}}
```

This keeps the two paths cleanly separated:
- Need everything in one tool? `azd ai project add connection` + azd env.
- Admin manages secrets externally? Create the connection in Portal or via `az` CLI, then reference it in azure.yaml with `${{}}` syntax.

### `azd ai project add model`

```
azd ai project add model [flags]
```

`` is the deployment name -- it maps to the `name` field on the [`ModelDeployment` struct](https://github.com/Azure/azure-dev/blob/main/cli/azd/extensions/azure.ai.agents/internal/project/config.go) and becomes the key in azure.yaml's `deployments:` list.

Adds a model deployment to the project.

```
$ azd ai project add model gpt-4.1-mini

? Model format: OpenAI
? Model name: gpt-4.1-mini
? Model version: 2025-04-14
? SKU name (Standard, GlobalBatch): Standard
? SKU capacity: 10

[x] Added model deployment 'gpt-4.1-mini' to project config.
Updated: azure.yaml

Next steps:
- Run 'azd deploy' to create the deployment in Foundry.
- Reference this deployment in your agent code:
AZURE_AI_MODEL_DEPLOYMENT_NAME=gpt-4.1-mini
```

### `azd ai project add skill`

```
azd ai project add skill [flags]
```

`` is the skill name -- it becomes the key in azure.yaml's `skills:` section.

Skills are reusable capabilities that bundle instructions, tools, and knowledge into a named unit. This is a new azure.yaml schema primitive, borrowing from Claude's Skills concept.

```
$ azd ai project add skill code-review \
--description "Reviews code for bugs and style issues" \
--instructions ./prompts/code-review.md

[x] Added skill 'code-review' to project config.
Updated: azure.yaml

Next steps:
- Edit the skill instructions at ./prompts/code-review.md
- Skills can reference tools already in the project.
- Run 'azd deploy' to push the updated config.
```

Skills in azure.yaml would look like:

```yaml
# Inside the azure.ai.project config block
skills:
code-review:
description: "Reviews code for bugs and style issues"
instructions: ./prompts/code-review.md
tools: [file-search, code-interpreter]
```

**Note:** Skills require a new section in the `azure.ai.project` config schema. This RFC covers the CLI surface; the schema design is a separate conversation.

---

## Config Modification

All `add` commands share a common engine that:

1. **Reads** and parses the `azure.ai.project` config block from `azure.yaml`.
2. **Validates** the addition -- type checks, duplicate detection, connection existence.
3. **Merges** the new entry into the config structure.
4. **Writes** the updated `azure.yaml`, preserving comments and formatting where possible.
5. **Externalizes credentials** to the azd environment, writing `${ENV_VAR}` references in the YAML.
6. **Prints** a summary of what changed, what files were touched, and what the developer needs to do next.

We reuse what the extension already has:

- Credential externalization follows the `externalizeCredentials()` pattern from init.
- Connection categories and auth types map to the enums already defined in the [`ConnectionResource`](https://github.com/Azure/azure-dev/blob/main/cli/azd/extensions/azure.ai.agents/internal/pkg/agents/agent_yaml/yaml.go) struct (`CategoryKind`, `AuthType`).

### Target Resolution Rules

The engine needs deterministic rules for picking which `azure.ai.project` service to modify:

1. **Project service resolution.** Scan `azure.yaml` for services with `host: azure.ai.project`. If exactly one exists, use it. If zero exist, add it and use it.

2. **Name collision.** If the name matches an existing connection, the command is a no-op: "Connection 'my-conn' already exists in project config. No changes made." No silent overwrites, no merging.

### Edge Cases and Error Behavior

| Scenario | Behavior |
|---|---|
| No `azure.yaml` in current directory | Error with suggestion: "No azure.yaml found. Run 'azd ai agent init' or specify --project-file." |
| `azure.yaml` exists but no `azure.ai.project` service | Valid scenario if the user used an existing project and no other project resources. |
| Multiple `azure.ai.project` services | Error, Unsupported scenario. |
| `--no-prompt` with missing required flags | Error listing the missing flags. No partial writes. |

---

## Interaction with `init`

These commands complement each other:

- **`init`** creates the project from scratch -- the `azure.ai.project` service, the first agent service, starter code, baseline config.
- **`add`** modifies the project after init. Minute five onward -- when you realize you need something you didn't pick initially.

`init` may use the `add` engine internally to populate initial config, keeping both paths consistent.

---

## Downstream Impact

- **`azure.ai` namespace:** Gets a new command subtree (`azd ai project add`)
- **azure.yaml schema:** Skills need a new config section in `azure.ai.project`.
- **Foundry Toolkit for VS Code:** Benefits indirectly. azure.yaml gets more complete as people use `add` instead of hand-editing.
- **Coding agents (Copilot skills):** `--no-prompt` and `--output json` make `add` commands callable from non-interactive agents. A Copilot skill can run `azd ai project add connection my-conn --category RemoteTool --no-prompt` as part of a larger workflow.
- **Samples and docs:** Walkthroughs can use `add` commands instead of "copy this YAML block" instructions. More durable, harder to get wrong.

---

## Scope Boundaries

**In scope for this RFC:**
- `azd ai project add` command family (connection, model, skill)
- Config modification engine (read, validate, merge, write azure.yaml)
- Credential externalization for new connections
- Interactive and non-interactive (`--no-prompt`) modes
- Structured output (`--output json`) for coding agent consumption
- Idempotency behavior

**Out of scope:**
- `azd ai remove` commands
- Code generation or modification (coding agents' domain)
- Bicep/infra generation (separate RFC)
- The `azure.ai.project` restructuring itself (covered by [#7962](https://github.com/Azure/azure-dev/issues/7962))
- Multi-project scenarios (one azure.yaml = one Foundry project)

---

## Remove / Delete Semantics

Removing a resource from azure.yaml does **not** delete it from Foundry. This matches how Bicep and azure.yaml already work -- dropping a resource from config means "stop using this," not "destroy it." That's the right default because project resources (toolboxes, connections, model deployments) can be shared across agents. Deleting a toolbox just because one agent's config no longer references it would break other agents that still use it.

The lifecycle for removal:

1. **Remove from config.** Developer deletes the entry from azure.yaml (hand-edit for V1). Next `azd deploy` pushes the updated agent definition without that resource reference. The resource itself remains in Foundry, untouched.

2. **Delete the resource.** If the developer actually wants to destroy the resource in Foundry, they use the `az` CLI (e.g., `az cognitiveservices account project connection delete`). Whether `azd` should wrap these delete operations is a separate RFC.

3. **Tear down everything.** `azd down` destroys the entire project and its resources. This is the nuclear option -- all-or-nothing today.

**Future work:** `azd down` should support targeted teardown of individual resources (e.g., `azd down --service foundry-project --resource connection:my-conn`) so developers don't have to choose between "delete nothing" and "delete everything." Tracking this separately.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.