microsoft / microsoft/hve-core

fix(agents,instructions,prompts,skills): convert remaining workspace-root `.github/` paths in distributed artifacts and add portability lint

Open
#2,607 0 comments 1 reaction 1 assignee Claimed by @jkim323 View on GitHub
agents bug infrastructure instructions prompts
Dominant language
Python
Stars
1.5k
Forks
301
Avg merge
3d 3h
Merged PRs (30d)
92

Description

## Summary

Refreshed scope and execution plan for the defect first reported in #741. Distributed AI artifacts still reference workspace-root-relative `.github/` paths in body prose. Those paths resolve against the workspace root, so they break when an artifact loads from a VS Code extension install directory, a peer clone, or a CLI plugin directory.

This issue supersedes the scope table in #741 for execution purposes. #741 remains the originating record.

Reference: #741

## Why a refreshed issue

Two things drifted since #741 was filed on 2026-02-23.

1. **The convention file named in #741 no longer exists.** #741 cites `prompt-builder.instructions.md` sections "Path Portability" (~L473) and "Skill File References" (~L325). The authoring standard was consolidated into `.github/instructions/hve-core/hve-builder.instructions.md`. Its functional successor is the `## Referencing Other Artifacts` section.
2. **The inventory is stale by more than half.** #741 lists 33 files and ~88 instances. After the package-subdirectory reorganization the current count is 47 files and 178 violations.

## Current inventory

Scanned 569 distributed artifacts. Root-level repo-specific artifacts and `deprecated/` paths are excluded, matching the distribution boundary already implemented by `Get-ArtifactFiles` in `scripts/lib/Modules/ArtifactHelpers.psm1`.

| Artifact type | Files scanned | Files with violations | LOAD | REF | Total |
|---|---|---|---|---|---|
| `*.agent.md` | 61 | 16 | 68 | 24 | 92 |
| `*.instructions.md` | 73 | 14 | 4 | 31 | 35 |
| `*.prompt.md` | 66 | 4 | 5 | 6 | 11 |
| `SKILL.md` | 57 | 3 | 0 | 13 | 13 |
| skill `references/` | 312 | 10 | 0 | 27 | 27 |
| **Total** | **569** | **47** | **77** | **101** | **178** |

`LOAD` means the AI is instructed to read the file at that path. `REF` means the path identifies an artifact where a name-based or filename-based reference would work.

A further ~161 instances are sanctioned and stay as-is: 66 installer install-targets, 8 subagent dispatch globs, 4 self-references, and ~83 descriptive.

### Highest-density files

| File | Violations |
|---|---|
| `.github/agents/project-planning/adr-creation.agent.md` | 25 (all LOAD) |
| `.github/agents/security/sssc-planner.agent.md` | 19 |
| `.github/agents/design-thinking/dt-coach.agent.md` | 17 (all LOAD) |
| `.github/instructions/privacy/privacy-identity.instructions.md` | 9 |
| `.github/agents/security/security-planner.agent.md` | 7 |
| `.github/skills/shared/backlog-templates/SKILL.md` | 7 |
| `.github/prompts/design-thinking/dt-canonical-deck.prompt.md` | 7 |
| `.github/skills/accessibility/accessibility/references/phases/framework-selection.md` | 7 |
| `.github/instructions/experimental/mural/mural-log-hygiene.instructions.md` | 6 |

`adr-creation.agent.md` is the clearest internal inconsistency: it already uses the correct `#file:../../instructions/...` form in four places while using workspace-root paths throughout its Lifecycle Dispatch tables.

## Acceptance criteria

- [ ] `hve-builder.instructions.md` names the bare `.github/`-prefixed prose path as prohibited, states the packaging rationale for agents, prompts, instruction files, and skills alike, and enumerates the sanctioned exceptions. (Retargets AC1 of #741 from the file that no longer exists.)
- [ ] Zero LOAD and zero REF instances remain across the 47 inventoried files, except instances explicitly allowlisted by a recorded decision.
- [ ] Every remaining operational reference uses one of the five portable forms below.
- [ ] All 10 `#file:` directives that carry a `.github/` prefix are converted, and no replacement crosses a marketplace package boundary.
- [ ] `hve-core-location.instructions.md` triggers on path presence, not only on read failure.
- [ ] The installer skill exclusion is stated in the authoring standard and encoded in the lint.
- [ ] `npm run lint:artifact-portability` exists, joins `validate:local`, and reports zero violations.
- [ ] `npm run validate:local` and `npm run test:ps` pass, with no content-assertion Pester guard weakened.

## Remediation vocabulary

Six patterns cover essentially all 178 violations.

| Form | Use for |
|---|---|
| `name-ref` | Skills, agents, subagents, prompts. Backticked frontmatter `name:`. Let progressive disclosure load it. |
| `filename-ref` | Instruction files, which have no `name:`. Bare `.instructions.md` plus section. |
| `file-directive` | Relative `#file:` with no `.github/` prefix. |
| `skill-relative` | Name the owning skill, then cite the resource relative to the skill root. |
| `semantic-invoke` | Script paths. Describe intent and let skill matching resolve. |
| `none` | Sanctioned. Leave unchanged. |

### `#file:` relative-depth reference

| From | Prefix |
|---|---|
| `.github/agents/{pkg}/x.agent.md` | `../../` |
| `.github/agents/{pkg}/subagents/x.agent.md` | `../../../` |
| `.github/instructions/{pkg}/x.instructions.md` | `../../` |
| `.github/prompts/{pkg}/x.prompt.md` | `../../` |
| same-directory sibling | `./` |

Two constraints govern `#file:` conversions:

- **Cross-package rule.** A `#file:` directive is legal only when its target ships in *every* marketplace package recipe that ships the referencing artifact. An artifact belonging to two recipes must satisfy both. Determine membership per file from `.github/plugin/marketplace.json`; do not infer a boundary from the directory name. Worked example: `rai-planner.agent.md` ships in both `rai-planning` and `data-science`, and `data-science` carries no mural instructions, so its mural references downgrade to `filename-ref`. `npm run lint:marketplace` is the gate.
- **Skill-target rule.** Never emit a `#file:` directive whose target lives inside a skill package, including `SKILL.md`, `SECURITY.md`, `references/`, `scripts/`, and `templates/`. Resolution of `#file:` into skill trees is unverified. Use `name-ref` plus `skill-relative`.

## Proposed work breakdown

Convention lands first so every rewrite cites one standard. The four conversion groups touch disjoint file sets and can run in parallel. Enforcement lands last so the new lint is born green rather than needing a warn-mode intermediate.

| Group | Scope | Violations / Files | Parallel |
|---|---|---|---|
| 1 | Convention foundation: `hve-builder.instructions.md` and `hve-core-location.instructions.md` | — | No |
| 2 | Design Thinking + Project Planning | 60 / 9 | Yes |
| 3 | Security + Supply Chain | 36 / 6 | Yes |
| 4 | Accessibility + Privacy + RAI | 44 / 14 | Yes |
| 5 | Shared, Experimental, Jira, cross-package | 38 / 18 | Yes |
| 6 | Ten judgment calls | 22 deferred | No |
| 7 | `lint:artifact-portability` script, Pester test, `package.json` wiring | — | No |
| 8 | Full validation and regeneration | — | No |

### Enforcement design

- New script `scripts/linting/Test-ArtifactPathPortability.ps1`, importing `Get-ArtifactFiles` from `ArtifactHelpers.psm1` rather than re-implementing enumeration. That function already encodes exactly the distributed-artifact scope boundary and is unit-tested.
- npm command `lint:artifact-portability`, appended to `validate:local` after `lint:ai-artifacts`. Not `ci:*`, which would exclude it from the local aggregate.
- JSON log at `logs/artifact-portability-results.json`.
- Mirrored test at `scripts/tests/linting/Test-ArtifactPathPortability.Tests.ps1`, tagged `Unit`.
- Detection: verb-proximity as the primary rule plus a short declared allowlist. Verb-proximity alone misses bare table cells such as `sssc-planner.agent.md` L50-L57, the second-largest violation cluster.
- Carve-outs: matches containing `**`, anything under `.github/skills/installer/`, frontmatter values including `applyTo`, fenced code blocks, markdown link targets, and placeholder-token paths such as `` or `{package-id}`.

## Exclusions

Unchanged from #741, restated for completeness. These are not violations.

| Pattern | Reason |
|---|---|
| `applyTo` frontmatter globs | Glob patterns, not file references |
| Subagent dispatch globs (`.github/agents/**/name.agent.md`) | Explicitly sanctioned in `.github/copilot-instructions.md` |
| Regex patterns | Pattern matching, not path resolution |
| Documentation tables describing directory structure | Informational, not operational |
| Placeholder-token paths (``, `{package-id}`) | Not resolvable targets |
| Relative markdown links | Already resolve correctly |
| Diff header examples | Git output examples |
| Installer skill `.github/` references | Install-target paths by design |

## Open decisions

Recommendations noted; each is reversible.

1. **Skill `references/` and vally-tests evidence.** Include the 27 skill-reference conversions (they ship and are read at runtime); exclude the ~40 vally-tests conformance citations (meaningful only inside hve-core). *Recommended.*
2. **`state.json` provenance values.** Six instances write a path-shaped value into persisted state, for example `"source": ".github/skills/rai/rai-standards/SKILL.md"`. Keep them path-shaped and allowlist them rather than changing a persisted contract that unaudited readers may depend on. *Recommended.*
3. **Hook manifests.** `.github/hooks/**` is returned by `Get-ArtifactFiles` but omitted from the #741 scope table. Decide whether `hooks` stays in the lint's path alternation.
4. **`applyTo` globs missing `**/`.** `vex-generation.instructions.md` L3 and `untrusted-content-boundary.instructions.md` L3 match nothing once distributed. Two-line fix; recommend folding in here rather than deferring.

## Known non-mechanical items

Ten items need a judgment call rather than a pattern application:

- `vally-test-author.agent.md` L35 cites `rai-risk-classification.instructions.md`, which no longer exists. This is a broken reference independent of portability and should be fixed regardless.
- `state.json` and `noticeLog[].source` provenance values across RAI and accessibility artifacts.
- `accessibility-surface-inventory.agent.md` targets a JSON schema inside a skill `scripts/` tree from an agent with no skill-relative frame.
- `security-reviewer-formats/references/finding-formats.md` L41 sits inside an output-format example; rewriting changes the emitted finding schema.
- `documentation/references/conventions.md` L13 targets a root-level repo-specific instruction that never ships.
- `cd .github/skills/.../` shell examples in `customer-card-render` and `powerpoint`.
- `framework-selection.md` L24 mixes a real prefix with a placeholder token for user-imported skill packages.

## Blast radius

- `docs/reference/**` is not affected. Asset docs build from frontmatter and metadata, not body text. Confirm with `npm run docs:generate:check`.
- `plugins/` is generated and must never be staged. `npm run plugin:generate` is a freshness check only.
- Real regression risk is four content-assertion Pester guards that read the artifacts being rewritten: `Test-SsscContextPreservation.Tests.ps1`, `Test-RiskGridGrammar.Tests.ps1`, `Test-PlannerStartupBlocks.Tests.ps1`, `Test-PlannerStateSchemas.Tests.ps1`. Run them sequentially after conversion; parallel Pester runs race on shared `logs/pester-*.json`.
- `evals/` hits are harness configuration and stimulus text, not artifact bodies. Low risk.

## Out of scope

Tracked separately:

- Scan `.github/skills/**/SECURITY.md`, `**/README.md`, and `.github/hooks/**` for the same defect class. One dangling reference already confirmed at `vex/SECURITY.md` L235.
- Repository-wide audit of `applyTo` globs for the missing `**/` failure mode, beyond the two confirmed.
- Verify whether `#file:` resolves from inside `SKILL.md` and skill `references/` files.
- Re-verify that `docs/templates/` and `scripts/` path portability is genuinely closed by #1334 and #1335.

## Related

- #741 (originating issue, milestone v3.12.0)
- #1334 and #1335 (portability audit for `scripts/`, `npm run`, and `docs/templates/` paths; explicitly left the bare `.github/` scope to #741)
- #1286 (established the template fallback pattern for code-review agents)
- #742, #767 (closed, adjacent reference-resolution fixes)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.