Azure / Azure/azure-dev

Support first-class data seeding workflow in azd

Open
#1,701 2 comments 1 reaction 1 assignee Assigned to @kristenwomack View on GitHub
area/hooks feature pm
Dominant language
Go
Stars
569
Forks
364
Avg merge
2d 19h
Merged PRs (30d)
136

Description

## Problem

After `azd provision` completes but before `azd deploy`, many real-world applications need a **data seeding step** — loading initial data into databases, uploading training data to ML services, configuring AI models, or populating storage accounts. Today this requires users to manually wire up `postprovision` hooks with custom scripts, which creates friction and inconsistency across templates.

### Common Scenarios

1. **Database seeding** — Run `.sql` scripts against a newly provisioned SQL/Postgres/Cosmos database to load schema + seed data
2. **AI/ML model setup** — Upload training data to Azure ML, configure AI model deployments, seed vector search indexes
3. **Blob/storage initialization** — Upload static assets, config files, or reference data to storage accounts
4. **Configuration seeding** — Populate App Configuration, Key Vault secrets, or feature flags with initial values

## Current State (What Works Today)

azd **named hooks** (`postprovision`, `postdeploy`, etc.) partially address this. Users can define hooks in `azure.yaml`:

```yaml
hooks:
postprovision:
- run: scripts/seed-database.sh
- run: scripts/upload-training-data.py
```

External hooks can also be defined via `main.hooks.yaml` in the infra path.

**This works** but requires every template author to:
- Write custom seeding scripts from scratch
- Manually wire hooks in azure.yaml
- Handle connection strings, auth, and error handling per script
- No discoverability — new users don't know this pattern exists

## What's Still Missing

1. **Convention-based discovery** — No automatic detection of seed data. If `./data/seed.sql` exists, azd doesn't know about it without explicit hook wiring
2. **First-class `azd seed` command** — No dedicated command to run seeding independently of the provision/deploy lifecycle (useful for re-seeding, dev reset scenarios)
3. **Built-in runners for common formats** — No native support for running `.sql` files against provisioned databases, uploading blobs to storage, or other common seeding patterns without writing a shell script
4. **Idempotency/state tracking** — No built-in way to track whether seeding has already run, skip re-seeding on subsequent `azd up` runs, or force re-seed

## Proposed Design

### Option A: Convention-Based Auto-Hooks (Low Effort)

Look for well-known files by convention and auto-register them as `postprovision` hooks:

| Convention | Behavior |
|-----------|----------|
| `data/seed.sql` | Run against database connection from provisioning output |
| `data/seed.py` | Execute Python script with env vars from provisioning |
| `data/seed.sh` / `data/seed.ps1` | Execute shell script |
| `data/*.json` or `data/*.csv` | Upload to blob storage (if storage account in outputs) |

### Option B: First-Class `azd seed` Command (Medium Effort)

Add `azd seed` as a lifecycle command between provision and deploy:

```yaml
seed:
database:
type: sql
source: ./data/schema.sql
target: ${AZURE_SQL_CONNECTION_STRING}
storage:
type: blob
source: ./data/assets/
target: ${AZURE_STORAGE_ACCOUNT_NAME}/assets
```

### Option C: Extension-Based (Aligned with Extension Framework)

Build seeding as an azd extension (e.g., `microsoft.azd.seed`) that provides the `azd seed` command and built-in runners, keeping core azd lightweight.

## Design Decisions Needed

- [ ] Convention-based auto-hooks vs. explicit `azd seed` command vs. extension?
- [ ] Which common formats deserve built-in runners (SQL, blob upload, etc.) vs. staying script-based?
- [ ] How to handle auth — reuse provisioning output env vars? Managed identity?
- [ ] Idempotency — should azd track seeding state, or leave that to the scripts?
- [ ] Should `azd up` include seeding automatically, or require explicit `azd seed`?

## Related

- Named hooks (postprovision, postdeploy) — current workaround
- `main.hooks.yaml` external hook files — convention for infra-path hooks
- Extension framework (#6853, #7520) — potential home for seed runners

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.