microsoft / microsoft/GitHub-Copilot-for-Azure

Replace microsoft-foundry skill dataset blob download-and-cache with a script

Open
#2,535 1 comment 0 reactions 1 assignee Claimed by @tmeschter View on GitHub
microsoft-foundry skills untriaged
Dominant language
Python
Stars
250
Forks
204
Avg merge
1d 12h
Merged PRs (30d)
67

Description

## Summary

Copilot has identified a portion of a skill that is a good candidate for replacement with a script.

The candidate is the **dataset blob download-and-cache** flow in the `microsoft-foundry` skill (`foundry-agent/observe/`) — a SAS → list-blobs → download-each → update-stub sequence with explicit failure-mode handling.

## Candidate description

To materialize a Foundry-managed eval dataset locally, the skill runs a fixed REST sequence with a loop:

1. Take a container-scope SAS URL (from the `evaluation_dataset_sas_url_get` MCP tool).
2. `GET ?restype=container&comp=list&` and parse blob names from the XML `EnumerationResults.Blobs.Blob.Name`.
3. Download **every** blob to a fixed `.foundry/datasets/-v/` path via `curl.exe`.
4. Write `contentDownloaded` / `contentPath` / `contentFiles` into the `.ref.json` stub.

This is a strong script candidate because it is:
- **Well-defined and looped** — explicit URL templates, a per-blob download loop, and a fixed output path.
- **Output-heavy** — XML listing where only blob names matter.
- **Platform-specific and brittle** — must use `curl.exe` (not PowerShell `Invoke-*`, which breaks on the URI), and the doc enumerates failure modes (PowerShell URI-parser breakage, `InvalidAuthenticationInfo`) a script would encapsulate once.
- **Multi-caller** — invoked from both `evaluation-suite-generation.md` and `deploy-and-setup.md` (L65).

**Sketch — `download-dataset-blobs.{sh,ps1}`:**
- **Input:** `--sas-url`, `--target-dir`.
- **Output:** the list of downloaded blobs and the updated stub, with a "downloaded N files to " summary.

> Only the upstream MCP call `evaluation_dataset_sas_url_get` and the host-pattern decision (Foundry-managed vs customer-owned connection) need agent context. The list → loop-download → stub-update body is fully mechanical; the script takes the SAS URL + target dir as inputs.

## Affected file and lines

- [`foundry-agent/observe/references/evaluation-suite-generation.md` — dataset content download (L66–L86)](https://github.com/microsoft/GitHub-Copilot-for-Azure/blob/3890cbfb65c548ce8daa96cabd1d8de63f7bbcca/plugin/skills/microsoft-foundry/foundry-agent/observe/references/evaluation-suite-generation.md#L66-L86)
- [`foundry-agent/observe/references/deploy-and-setup.md` — materialize dataset blobs (L65)](https://github.com/microsoft/GitHub-Copilot-for-Azure/blob/3890cbfb65c548ce8daa96cabd1d8de63f7bbcca/plugin/skills/microsoft-foundry/foundry-agent/observe/references/deploy-and-setup.md#L65)

## Next steps

1. **Evaluate the candidate** — confirm the steps are stable and parameterizable, and that the script captures everything the skill needs.
2. **Create both a bash _and_ a PowerShell version** of the script so the skill works across platforms.
3. **Run integration tests** to verify the scripts behave correctly and the skill still completes end-to-end.

## Background Information

### Why replace regular steps with scripts

Replacing a regular, well-defined series of steps with a script can:

- **Reduce token usage** — the skill no longer needs to spell out each command and parse large command output inline; the agent invokes one script and reads a compact result.
- **Improve reliability** — the logic is written and tested once, instead of being re-derived by the agent on every run.
- **Improve determinism** — the same inputs always produce the same steps and output, removing run-to-run variation.
- **Improve speed of execution** — a single script call replaces multiple round-trips of command generation, execution, and large-output parsing.

### Authoring notes for the scripts

- **Reference scripts with markdown links**, not just a bare path to the script file.
- **Include examples** in the skill showing how to run each script (sample invocation with arguments).
- **Briefly explain what each script does** where it is referenced.
- **The script output should explain what it did**, so the agent and user can understand the result without re-inspecting raw command output.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.