[New Feature]: Request Copilot review for New-Package metadata

Description of the new feature/enhancement

Automatically request a GitHub Copilot code review when a pull request is deterministically classified as a New-Package submission.

The review should compare only the non-omitted metadata in the changed default-locale manifest against authoritative publisher-controlled sources. This addresses metadata quality that schema and installer validation cannot determine, without adding another bot comment or running semantic review on every manifest update.

Part of the workflow family tracked in #411014.

Feedback and PPE evaluation results are collected in the Agent Workflows discussion:
https://github.com/microsoft/winget-pkgs/discussions/411359

Pilot findings

A pilot reviewed the exact pull request head manifests from ten merged community New-Package submissions:

• https://github.com/microsoft/winget-pkgs/pull/410965
• https://github.com/microsoft/winget-pkgs/pull/409656
• https://github.com/microsoft/winget-pkgs/pull/409637
• https://github.com/microsoft/winget-pkgs/pull/408829
• https://github.com/microsoft/winget-pkgs/pull/408034
• https://github.com/microsoft/winget-pkgs/pull/407683
• https://github.com/microsoft/winget-pkgs/pull/407624
• https://github.com/microsoft/winget-pkgs/pull/407568
• https://github.com/microsoft/winget-pkgs/pull/407017
• https://github.com/microsoft/winget-pkgs/pull/406867

The sample demonstrated why the review must be high precision:

• Most supplied metadata was accurate or an acceptable publisher-aligned paraphrase.
• Two packages lacked sufficient authoritative public sources, so the correct result was no finding, not speculation.
• Several initially plausible findings disappeared after correcting the analysis to use the exact PR head rather than a later repository state.
• One likely actionable contradiction remained: MeldStudio.MeldStudio declared License: Proprietary (Requires account), while official Meld documentation states that streaming does not require an account login and account connection is feature-specific.
• No sampled ShortDescription used a generic placeholder, but a deterministic placeholder check remains useful for values such as New Package Setup, Setup, or a value that merely repeats the package name.

Trigger investigation update

The initial proposal requested Copilot review as soon as the PR received the
New-Package classification. Label timing shows that a more efficient design
needs to distinguish evidence collection from semantic review:

• In representative PRs, New-Package was normally added about 14-24 minutes
  after submission.
• Clean validation completion or a failure reason generally arrived about
  50-150 minutes after submission.
• Difficult cases can take much longer. PR #410965 did not receive its final
  classification and validation outcome until after repeated validation runs.

This creates two useful decision points:

1. Submission/classification preflight: collect cheap deterministic evidence
   without requesting Copilot review or notifying the author.
2. Validation outcome: route the PR based on the labels that explain what
   happened, and request semantic metadata review only when it remains useful.

Proposed two-stage behavior

Stage 1: deterministic preflight

On PR open/synchronize, or when New-Package / transitional New-Manifest is
added:

• Parse the PR body and changed files.
• Confirm the PR contains one package/version and identify the exact head
  locale manifest.
• Record author/bot classification and whether the PR is ESRP/Verified
  Developer automation.
• Inspect any changed .validation file and relevant package-root waiver
  metadata without treating it as automatically applicable.
• Record static publisher/package/installer domain relationships for later
  label-driven assessment.
• Do not comment, request review, or add labels.

Waiver files are version- and commit-scoped JSON records normally created by
wingetbot in a separate PR. Community-authored PRs that add .validation files
are already marked Author-Not-Authorized. Therefore, waiver presence is useful
submission evidence, but it is not a general bypass and is unlikely to apply to
a genuinely new package.

Stage 2: outcome-driven review

When labels are added:

• Validation failure label: use the failure-specific workflow and combine
  the label/log reason with the preflight evidence. Do not also request a broad
  metadata review.
• Validation-Completed + New-Package / New-Manifest: this is the
  preferred deferred trigger for the publisher-metadata review pilot. The PR
  has survived deterministic validation, remains in the human-review lane, and
  is less likely to require another author push that would stale the review.
• Security labels: never request semantic review or attempt remediation.
• ESRP/Verified Developer PRs: exclude unless product owners explicitly opt
  them in.

Request copilot-pull-request-reviewer[bot] at most once per head commit. The
existing .github/instructions/manifests.instructions.md should be extended
with the publisher-metadata review rules below. A separate agent-authored
commenting workflow is not needed for v1.

Review scope

For the changed default-locale manifest, review only fields that are present:

• ShortDescription
• Description
• License
• ReleaseNotes
• ReleaseNotesUrl
• Documentations

Use explicit PackageUrl, PublisherUrl, and publisher-controlled repository links as the source-discovery boundary. For GitHub-hosted projects, prefer the exact-version release, README, license file, and official documentation.

High-confidence findings

Copilot should report only:

• Generic or placeholder ShortDescription text that does not identify the product.
• A factual contradiction between supplied metadata and an authoritative publisher source.
• A supplied license that conflicts with an explicit publisher license or EULA.
• Supplied release notes that contradict the exact submitted version's publisher release.
• A supplied documentation or release-notes URL that identifies the wrong product or version.

Mandatory no-finding cases

• Optional fields are omitted.
• The wording is an accurate paraphrase.
• The description omits secondary features but remains true.
• A publisher source is unavailable or ownership cannot be established.
• The evidence is ambiguous, incomplete, or only available from a third-party source.
• The concern is style, field ordering, punctuation, SPDX formatting, or description length.
• A release-notes URL points to a relevant publisher release index rather than a version-specific page.

Guardrails

• Review only the files changed by the PR and the immediate package folder.
• Never recursively search manifests/.
• Never download or execute installers.
• Never infer facts from unrelated mirrors, aggregators, or third-party reviews.
• Never post a separate bot comment, request changes, add labels, or block merging automatically.
• ESRP/Verified Developer auto-merge PRs remain excluded unless product owners explicitly opt them in.

PPE validation

Copy representative production New-Package submissions into microsoft/winget-pkgs-preprod, request Copilot review, and measure:

• actionable findings;
• false positives;
• no-finding rate;
• source-verification failures;
• review latency and GitHub Actions consumption.

Test two trigger cohorts:

1. Request review when New-Package is first added.
2. Request review only after both New-Package and Validation-Completed are
   present.

Compare how often the early review becomes stale after author updates and how
much latency the deferred trigger adds. Start with the ten pilot packages above
plus a known generic-description example. Do not enable production automation
until the PPE review results are manually evaluated.

Created with GitHub Copilot assistance.

Start by reading .github/instructions/manifests.instructions.md and the workflow family tracked in #411014, then review the ten pilot pull requests and the Agent Workflows discussion. Done means the two-stage trigger, metadata guardrails, and PPE comparison are defined and manually evaluated before any production automation is enabled.