Azure / Azure/azure-dev

Add an azd ai dataset extension for Foundry datasets

Open
#9,550 0 comments 0 reactions 0 assignees View on GitHub
area/evals enhancement
Dominant language
Go
Stars
569
Forks
364
Avg merge
2d 19h
Merged PRs (30d)
136

Description

### What

`azd` has no way to create or version Foundry datasets. A dataset is uploaded by
hand against the data plane, so the evaluation that grades against it cannot be
described in the project at all.

### Proposal

Add an `azd ai dataset` extension that registers a dataset and publishes its
versions, so a dataset is declared alongside the rest of the project and
published by the same deploy that publishes everything else.

Companion to #9549, which adds the `azd ai eval` extension that consumes them.

Contributor guide

Open the contributing guide

Research direction

Start by reading the existing azd ai command extensions and the companion issue #9549 to understand how Foundry evaluations are represented. The work is done when a dataset can be declared, registered, and have its versions published through the same deploy that publishes the rest of the project.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure
Domain
ai, cli
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.