microsoft / microsoft/GitHub-Copilot-for-Azure
Replace microsoft-foundry skill dataset blob download-and-cache with a script
- Dominant language
- Python
- Stars
- 250
- Forks
- 204
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 67
Description
## Summary
Copilot has identified a portion of a skill that is a good candidate for replacement with a script.
The candidate is the **dataset blob download-and-cache** flow in the `microsoft-foundry` skill (`foundry-agent/observe/`) — a SAS → list-blobs → download-each → update-stub sequence with explicit failure-mode handling.
## Candidate description
To materialize a Foundry-managed eval dataset locally, the skill runs a fixed REST sequence with a loop:
1. Take a container-scope SAS URL (from the `evaluation_dataset_sas_url_get` MCP tool).
2. `GET ?restype=container&comp=list&` and parse blob names from the XML `EnumerationResults.Blobs.Blob.Name`.
3. Download **every** blob to a fixed `.foundry/datasets/-v/` path via `curl.exe`.
4. Write `contentDownloaded` / `contentPath` / `contentFiles` into the `.ref.json` stub.
This is a strong script candidate because it is:
- **Well-defined and looped** — explicit URL templates, a per-blob download loop, and a fixed output path.
- **Output-heavy** — XML listing where only blob names matter.
- **Platform-specific and brittle** — must use `curl.exe` (not PowerShell `Invoke-*`, which breaks on the URI), and the doc enumerates failure modes (PowerShell URI-parser breakage, `InvalidAuthenticationInfo`) a script would encapsulate once.
- **Multi-caller** — invoked from both `evaluation-suite-generation.md` and `deploy-and-setup.md` (L65).
**Sketch — `download-dataset-blobs.{sh,ps1}`:**
- **Input:** `--sas-url`, `--target-dir`.
- **Output:** the list of downloaded blobs and the updated stub, with a "downloaded N files to " summary.
> Only the upstream MCP call `evaluation_dataset_sas_url_get` and the host-pattern decision (Foundry-managed vs customer-owned connection) need agent context. The list → loop-download → stub-update body is fully mechanical; the script takes the SAS URL + target dir as inputs.
## Affected file and lines
- [`foundry-agent/observe/references/evaluation-suite-generation.md` — dataset content download (L66–L86)](https://github.com/microsoft/GitHub-Copilot-for-Azure/blob/3890cbfb65c548ce8daa96cabd1d8de63f7bbcca/plugin/skills/microsoft-foundry/foundry-agent/observe/references/evaluation-suite-generation.md#L66-L86)
- [`foundry-agent/observe/references/deploy-and-setup.md` — materialize dataset blobs (L65)](https://github.com/microsoft/GitHub-Copilot-for-Azure/blob/3890cbfb65c548ce8daa96cabd1d8de63f7bbcca/plugin/skills/microsoft-foundry/foundry-agent/observe/references/deploy-and-setup.md#L65)
## Next steps
1. **Evaluate the candidate** — confirm the steps are stable and parameterizable, and that the script captures everything the skill needs.
2. **Create both a bash _and_ a PowerShell version** of the script so the skill works across platforms.
3. **Run integration tests** to verify the scripts behave correctly and the skill still completes end-to-end.
## Background Information
### Why replace regular steps with scripts
Replacing a regular, well-defined series of steps with a script can:
- **Reduce token usage** — the skill no longer needs to spell out each command and parse large command output inline; the agent invokes one script and reads a compact result.
- **Improve reliability** — the logic is written and tested once, instead of being re-derived by the agent on every run.
- **Improve determinism** — the same inputs always produce the same steps and output, removing run-to-run variation.
- **Improve speed of execution** — a single script call replaces multiple round-trips of command generation, execution, and large-output parsing.
### Authoring notes for the scripts
- **Reference scripts with markdown links**, not just a bare path to the script file.
- **Include examples** in the skill showing how to run each script (sample invocation with arguments).
- **Briefly explain what each script does** where it is referenced.
- **The script output should explain what it did**, so the agent and user can understand the result without re-inspecting raw command output.
Contributor guide
Assessment
This issue has not been assessed yet.