Encapsulate vMCP assembly behind a config-driven public API

Summary / Motivation

vMCP's assembly — turning configuration into a running server — lives in
pkg/vmcp/cli/serve.go (~700 lines). It builds backend discovery (the dynamic
BackendRegistry + Kubernetes watcher, or static/inline backends), the aggregator,
router, backend client, health monitor, optimizer, session-manager config,
incoming-auth and rate-limit middleware, the workflow definitions, and finally the
server.

That wiring is not packaged. An embedder (e.g. the enterprise gateway) that wants
a vMCP server must re-implement the same sequence by hand and honor assembly rules
that today exist only in doc comments — and it breaks whenever OSS changes how vMCP
assembles itself. The cost of keeping that copy in sync belongs in the OSS.

#5541 (merged via #5542) encapsulated one seam — the Kubernetes-backed
BackendRegistry + watcher — so embedders no longer hand-wire it. This issue finishes
the job for the assembly as a whole: a caller hands over config and gets back a
VMCP core (to decorate) plus the transport config to serve. The OSS owns the
assembly; embedders compose vMCP instead of reimplementing it.

Raised as a blocker on #5542:
https://github.com/stacklok/toolhive/pull/5542#issuecomment-4770537494

vMCP should encapsulate its own assembly. An embedder should hand over config and
get a server back — not wire up vMCP's internals or honor assembly rules that live
only in doc comments. […] which backends exist, and how they're discovered, is a
config choice — not a dependency the embedder constructs and injects.

This extends the direction of epic #5419 / RFC THV-0076. As written, THV-0076 keeps
assembly at the composition root and treats discovery as an injected dependency, so
this adds new public surface beyond the RFC — flagging for @jerm-dro and a possible RFC
follow-up before/with implementation.

Goal

Config in, server out — with decoration preserved (the RFC's supported extension
mechanism):

// Shared stateful collaborators (registry+watcher, telemetry, health) are built once
// and passed to BOTH derivations via opts; every other field each derives from vmcpCfg.
opts := []app.Option{app.WithVersion(v) /*, app.WithBackendRegistry(reg, watcher), ... */}

core, coreCleanup, err := app.BuildCore(ctx, vmcpCfg, opts...)
defer coreCleanup()

// the transport config is derived from vmcpCfg directly — NOT returned by BuildCore
serverCfg, srvCleanup, err := app.BuildServerConfig(ctx, vmcpCfg, opts...)
defer srvCleanup()

// decoration is the embedder's business, invisible to the server
srv, err := server.Serve(ctx, userScoped(core), serverCfg)

Dependency injection via core.New(&core.Config{...}) + server.Serve(ctx, v, &server.ServerConfig{...}) stays for tests and advanced embedders — both coexist.

Design (proposed)

• New OSS package — NOT root pkg/vmcp (it would import-cycle with core/server).
  Proposed pkg/vmcp/app (name TBD).
• Input is the existing serialized vmcpconfig.Config — already the single source
  of truth (Group, Backends, OutgoingAuth.Source: inline|discovered, aggregation,
  optimizer, incoming/outgoing auth, telemetry, audit, rate limiting, composite tools).
  No new parallel config type (avoids the go-style "parallel types that drift"
  anti-pattern). The assembly OWNS interpreting it — including choosing the discovery
  mechanism from OutgoingAuth.Source + Backends/Group — so the embedder no longer
  encodes those rules.
• Options carry the non-serialized runtime injectables that are CLI / composition-
  root concerns today: REST config (default in-cluster, reusing #5541's
  backendregistry.WithRESTConfig), container factory (TEI optimizer), embedded auth
  server, version string, host/port, and an injected *slog.Logger (cf. #5381).
• Two independent derivations from vmcpconfig.Config, mirroring the existing
  deriveCoreConfig/deriveServerConfig split (#5444):
  • BuildCore derives core.Config from vmcpConfig (+ opts) and calls core.New,
    returning only the VMCP (for decoration). It does not return a
    ServerConfig — the transport config is not a byproduct of core construction.
  • BuildServerConfig derives *server.ServerConfig from vmcpConfig (+ opts)
    directly — an independent projection of the same input.
    Both reuse the existing deriveCoreConfig / deriveServerConfig / WithDefaults
    helpers (pkg/vmcp/server/derive.go, promoted to exported or relocated), so there is
    exactly one config-split path, shared with #5445's reduced server.New.
• Shared stateful collaborators are built once and threaded into both derivations,
  never built twice: the TelemetryProvider (else duplicate OTEL pipelines), the
  *health.Monitor (else double health-check traffic / a divergent view), and — in
  dynamic mode — the BackendRegistry+watcher (else two informer caches). These are the
  override seams (Options): the caller (or the one-shot app.Serve) builds them once,
  passes the same opts to both calls, and owns their lifecycle/cleanup. Each Build*
  cleanup releases only what it acquired. This mirrors how deriveCoreConfig/
  deriveServerConfig already take backendRegistry/healthMon as explicit shared args.
• Reuse #5541 for the discovered path; build the static/inline registry for
  Backends.

Scope

• app.BuildCore(ctx, *vmcpconfig.Config, ...Option) (core.VMCP, cleanup func(), error)
  — derives core.Config from vmcpConfig (+ opts) and calls core.New. Returns
  only the core (no ServerConfig). Reuses the #5444 derive helpers and #5541's
  registry constructor.
• app.BuildServerConfig(ctx, *vmcpconfig.Config, ...Option) (*server.ServerConfig, cleanup func(), error)
  — derives the transport config directly from vmcpConfig (+ opts), reusing the
  #5444 derive helpers.
• Functional Options for the runtime injectables and the shared stateful
  collaborators (REST config, container factory, embedded auth server, telemetry
  provider, health monitor, backend registry+watcher, version, host/port, logger) —
  so the same opts passed to both Build* calls share one instance.
• Runnable example embedder: BuildCore + BuildServerConfig → decorate →
  server.Serve — extending the decorator example from #5447.
• Unit tests: each derivation yields a working core / ServerConfig for both
  discovered and inline discovery; options are applied; a shared collaborator
  passed to both is built once; each cleanup releases only what it acquired.
• (optional) one-shot convenience app.Serve(ctx, *vmcpconfig.Config, ...Option) (*server.Server, error)
  that builds the shared collaborators once, derives both configs, and serves — for
  the no-decoration case.
• Migrate pkg/vmcp/cli/serve.go to call the new assembly API (app.BuildCore +
  app.BuildServerConfig), replacing the hand-wired assembly in the OSS production
  serve path with a call through the new package. This is what actually removes the
  OSS duplication and proves the API is sufficient for the real caller. Coordinate
  with #5445 to ensure a single shared config-split path.

Out of scope (tracked as follow-ups)

• Enterprise gateway adopting the API — separate repo / issue.
• Retiring the legacy server.Config god-object and the deriveCoreConfig /
  deriveServerConfig helpers once both server.New and cli.Serve route through the
  assembly.
• Any change to the serialized vmcpconfig.Config schema / CRD / YAML.
• A new programmatic vmcp.Config sum-type (e.g. Discovery: KubernetesDiscovery | StaticDiscovery): considered and rejected for now in favor of reusing
  vmcpconfig.Config, to avoid maintaining a second config type.

Dependencies / coordination

• Builds on #5541 (registry seam) and the #5444 derive helpers (closed).
• Coordinate with #5445 (Phase 3 — reduces server.New to a wrapper over the same
  derive helpers): both must share a single config-split path.

References

• Blocker comment: https://github.com/stacklok/toolhive/pull/5542#issuecomment-4770537494
• Epic #5419; RFC THV-0076
• Related: #5432, #5433, #5444, #5445, #5447, #5541, #5381

Start with pkg/vmcp/cli/serve.go and pkg/vmcp/server/derive.go, then review the existing core/server construction and #5445's shared config-split path. Add the proposed app.BuildCore and app.BuildServerConfig API, cover discovered and inline modes, options, shared collaborators, and cleanup in unit tests, then migrate the CLI assembly to use it.