microsoft / microsoft/apm

[FEATURE] Global marketplace directories for enterprise-wide discovery

Open
#3,006 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area/cli area/docs-site area/enterprise area/marketplace status/deferred theme/security triage/recommended type/feature
Dominant language
Python
Stars
3.9k
Forks
365
Avg merge
1d 17h
Merged PRs (30d)
132

Description

Is your feature request related to a problem? Please describe.

Large organizations can have many teams that publish independent APM
marketplaces. Engineers currently need to know about each marketplace in
advance, register it locally, and search it by name:

apm marketplace add <source> --name <marketplace>
apm search <query>@<marketplace>

This process does not give an enterprise a standard method to publish the set
of marketplaces available across the organization. Discovery also depends on
per-developer registrations that can become stale.

Projects are not the correct scope for this directory. A project manifest
must continue to contain only the packages and agent assets selected for that
project. Enterprise marketplace discovery is user-level configuration that
applies across an engineer's projects.

Describe the solution you'd like

Add a versioned, flat marketplace-directory specification. Let the resolved
user-level APM manifest reference one or more marketplace directories.

1. Global-only consumer configuration

The user manifest, normally ~/.apm/apm.yml, can contain:

marketplaceDirectories:
  - id: enterprise-git
    source:
      type: git
      url: https://git.example.com/platform/apm-marketplaces.git
      ref: main
      path: marketplace-directory.json

  - id: enterprise-artifacts
    source:
      type: https
      url: https://artifacts.example.com/apm/marketplace-directory.json

APM must use the effective user configuration root instead of a hard-coded
~/.apm path. This behavior keeps existing configuration-root overrides
testable.

marketplaceDirectories is valid only in the user-level manifest. If it is
present in a project apm.yml, APM must produce an actionable validation
error. Project configuration must not add, replace, or override directory
sources.

The field contains discovery configuration only. APM must not resolve user
dependencies, run lifecycle content, or deploy content when it reads this
field.

2. Versioned marketplace-directory file

A directory is a strict JSON document:

{
  "schemaVersion": 1,
  "id": "enterprise",
  "marketplaces": [
    {
      "id": "mobile",
      "source": {
        "type": "git",
        "url": "https://git.example.com/mobile/apm-marketplace.git",
        "ref": "main",
        "path": ".claude-plugin/marketplace.json"
      }
    },
    {
      "id": "data",
      "source": {
        "type": "https",
        "url": "https://artifacts.example.com/apm/data/marketplace.json"
      }
    }
  ]
}

Version 1 must:

  • Support Git-hosted marketplace manifests through APM's existing Git
    transport and authentication rules.
  • Support direct anonymous HTTPS marketplace.json files.
  • Forbid credentials, authorization headers, cookies, user information,
    embedded tokens, HTTP URLs, local paths, and file: URLs.
  • Reject cross-origin redirects for anonymous HTTPS sources.
  • Reject unknown fields, path traversal, malformed identifiers, control
    characters, and oversized documents.
  • Limit one directory file to 1 MiB and 256 marketplace entries.
  • Require stable directory and marketplace identifiers with a documented,
    case-normalized identifier grammar.
  • Reject conflicting marketplace identifiers across the complete effective
    directory set.
  • Deduplicate an identical marketplace source declared by multiple
    directories. Reject one identifier that maps to different sources.
  • Remain flat. A marketplace-directory entry cannot reference another
    marketplace directory.

Publish a JSON Schema and conformance fixtures with the specification.

3. Dynamic, atomic refresh

Before each command that consumes marketplace-directory state, APM must
retrieve and validate the current directory files and referenced marketplace
manifests.

For the first version, the directory-consuming command is:

apm search QUERY

Each CLI invocation must see the latest directory content. Unrelated commands,
--help, and --version must not make directory network requests. This
feature does not require a daemon or background refresh.

For Git sources, APM must resolve the configured reference to the commit used
by that invocation. For HTTPS sources, APM can use ETag or Last-Modified
revalidation. APM must not silently treat stale cached data as current.

APM must construct one atomic snapshot before it displays results. A
transport, authentication, parsing, schema, size, identity, or marketplace
validation failure must return a nonzero exit status and no partial result
set. Fetch completion order must not affect result order.

APM can use a cache for performance, but the cache is not an offline fallback.
Cache metadata must record the source URL, resolved Git commit or HTTPS
validators, the SHA-256 digest of the fetched bytes, and the retrieval time.

The configured HTTPS origin is the trust anchor for a dynamic directory. A
required content hash in every developer's configuration would prevent a
central publisher from updating the directory automatically. APM can support
an optional sha256: constraint for organizations that require a static
directory.

In both modes, APM must report the observed digest in verbose or structured
output. APM must not represent an observed digest as a publisher signature.

4. Search and installation boundary

With directories configured, this command searches all marketplaces in the
current atomic snapshot:

apm search security

Existing behavior remains unchanged:

apm search security@skills

The qualified command continues to search the named marketplace in the local
marketplace registry. A bare query must not silently use that registry. If no
global directories exist, APM must explain how to configure one.

Results must use a deterministic order and identify:

  • The directory identifier.
  • The marketplace identifier.
  • The package name and description.
  • The canonical package source.
  • The resolved package commit.
  • A directly installable APM expression.

A directory search result must not depend on a stored package@marketplace
alias. Before APM prints an install expression, it must resolve a matching Git
package source to an immutable commit. The expression must use an existing
direct-install form that includes the source, commit, and package path.

If a matched entry cannot produce a replayable direct coordinate, APM must
report a validation error instead of a mutable install suggestion.

Installation remains a separate, explicit action:

apm install <direct-source-at-resolved-commit>

That command uses the normal APM resolution, audit, policy, approval, content
scan, lockfile, and deployment behavior.

A directory refresh or search must never:

  • Install, update, approve, remove, or deploy a package.
  • Write a marketplace alias to a project or user manifest.
  • Modify ~/.apm/marketplaces.json.
  • Write an APM lockfile.
  • Download or execute package content.
  • Grant trust to a marketplace or package.

APM must treat directory and marketplace metadata as untrusted display data.
It must reject or sanitize terminal control characters. Search results must
show the package source before the user installs it.

A directory change or removal must not affect installed packages. Frozen or
lockfile-based installation must not consult marketplace directories.

5. Publisher workflow

Provide a small authoring surface:

apm marketplace directory init marketplace-directory.json
apm marketplace directory validate marketplace-directory.json
apm marketplace directory validate marketplace-directory.json --resolve

init creates a deterministic, versioned local file. validate checks the
file and schema without network access. --resolve also checks the referenced
marketplace manifests and their direct installation coordinates.

These commands must not discover credentials, upload content, change user
configuration, or infer an integrity constraint from an uploaded URL.

Publishing remains external to APM:

  1. A maintainer creates or updates the directory file.
  2. APM validates the file.
  3. Existing Git or artifact-pipeline tooling uploads or commits the file.
  4. Consumers retrieve the file through Git or anonymous, read-only HTTPS.

APM must not add an artifact-repository upload API or a generic
directory publish command.

Done when
  • The effective user manifest can declare multiple directory sources.
  • APM rejects the same field in project manifests.
  • Git and anonymous HTTPS directory sources follow the documented transport
    rules.
  • apm search QUERY refreshes and searches the complete directory set.
  • apm search QUERY@MARKETPLACE and the local marketplace registry retain
    their current behavior.
  • Directory and marketplace refresh is atomic and produces no partial results
    after a failure.
  • Search output is deterministic and contains a replayable, immutable direct
    installation coordinate.
  • Search and refresh perform no installation, deployment, lifecycle execution,
    lockfile writes, or local marketplace-registry changes.
  • APM can create and validate directory files locally. External tooling remains
    responsible for upload.
  • Tests cover project-scope rejection, multiple directories, anonymous HTTPS
    requests, Git ref resolution, duplicate identities, traversal and size
    rejection, atomic failure, deterministic output, direct-install replay, and
    the absence of registry or installation side effects.
  • Documentation covers the global manifest field, search behavior, directory
    specification, authoring workflow, and security model.
Out of scope
  • Project-level marketplace directories.
  • Directories that contain other directories.
  • Automatic installation, synchronization, approval, or deployment.
  • Use of a directory as enterprise policy or enforcement.
  • Authenticated artifact-repository downloads.
  • Upload to an artifact repository or Git host.
  • Replacement of the existing local marketplace registry.
  • Storage of marketplace aliases in project or global dependency manifests.
  • Changes to existing lockfile replay semantics.
  • Background refresh or a long-running APM service.

Describe alternatives you've considered

  • Publish a central document and require engineers to register each
    marketplace manually. This process is not standard or toolable, and local
    registrations can become stale.
  • Put the complete marketplace list in each project. This approach mixes
    enterprise discovery with a project's explicit dependency set.
  • Merge all team marketplaces into one marketplace. This approach centralizes
    ownership and release coordination.
  • Let directories install or register marketplaces automatically. This
    approach turns untrusted discovery metadata into a state-changing operation.
  • Require a content hash for each changing HTTPS directory revision. This
    improves static-content integrity but prevents automatic central updates.
    Git-pinned and optional digest-pinned sources remain available for static
    use.

Additional context

As of v0.31.0, apm search requires QUERY@MARKETPLACE and searches one
marketplace from the local marketplace registry.

Related work:

  • #501 introduced marketplace discovery and governance.
  • #722 covered generation of one marketplace artifact.
  • #1136 covered curated marketplace pass-through. It republishes upstream
    content and differs from a flat discovery directory.
  • #2903 covers durable marketplace identities and catalog pins. This proposal
    does not require that design. Directory search emits direct, immutable
    package coordinates, and explicit installation uses the existing manifest
    and lockfile path.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The payload names no implementation files; begin with the apm search entry point and effective user/project manifest loading, then trace existing marketplace-registry and Git transport behavior. Review the directory schema, validation, refresh, and direct-install requirements, plus the listed related issues. Done means the stated search, authoring, security, atomicity, compatibility, and side-effect tests and documentation are covered.

Written by the indexing model from the issue text.

Assessment

Tech stack
git, json, python, yaml
Domain
cli, devtools, networking, security, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.