Measure model selections and approved provider profiles

## Outcome

Measure the distribution of recorded model selections, provider profiles, and configured inference API families across final applied agent-route assignments in published NemoClaw environments without transmitting arbitrary model IDs, provider names, endpoints, credentials, or inference activity.

This helps maintainers decide which model and provider combinations need compatibility testing, documentation, and support.

## Dependencies

- #10435 must define and receive approval for the telemetry privacy, service, schema, allowlist, and reporting contract.
- Extend the post-commit registry observation mechanism from #10442. Do not create a second lifecycle observer or delivery path.
- Implement only after an accepted mutating child of #10904 persists complete final applied multi-agent route assignments. Authored YAML and declared provider definitions are not telemetry sources.
- Keep this implementation last, after #10440, #10442, #10447, and #10448.
- The product and privacy owners must approve the checked-in model-key and provider-profile allowlists.

## OpenShell design alignment

OpenShell’s anonymous telemetry reports provider profiles through a compile-time allowlist. Recognized profiles map to product-owned categories, and every unrecognized value maps to `custom`.

OpenShell does not send model names through its anonymous community telemetry. Exact model names and endpoints appear only in its separate local OCSF observability records.

NemoClaw follows the anonymous telemetry pattern:

- Use closed provider categories.
- Map arbitrary provider values to `custom` or `unknown`.
- Send exact aggregate counts.
- Never send or hash a raw model or provider value.
- Attempt best-effort delivery without changing product behavior.

NemoClaw adds a checked-in model key and explicit selection-source category. These are safe only because their values are product-owned and closed; this issue does not authorize sending the recorded model ID.

## Measurement

Emit `nemoclaw_model_observed` with:

| Field | Allowed values |
| --- | --- |
| `model_source` | `product_catalog`, `provider_catalog`, `custom`, `local`, or `unknown` |
| `known_model_key` | A checked-in product-owned telemetry key, `other`, or `unknown` |
| `provider_profile` | `nvidia`, `openai`, `anthropic`, `google-gemini`, `openrouter`, `hermes`, `ollama`, `vllm`, `llama-cpp`, `compatible-endpoint`, `custom`, or `unknown` |
| `api_family` | `openai-completions`, `openai-responses`, `anthropic-messages`, or `unknown` |
| `count` | An exact positive integer |

Every event also uses the common envelope defined by #10435.

Project the complete final applied agent-route state for published sandboxes. Emit one event for each nonzero combination of the four categorical fields.

The observation unit is one durable route entry assigned to one configured agent after a successful apply. Each assignment contributes exactly once, including assignments whose recorded selection maps entirely to `unknown`.

If two agents reference the same provider definition or otherwise have identical resolved routes, count two assignments. If one agent has two routes, count two assignments. An unused provider or route definition, an agent with no route, and a sandbox with no route contribute zero and produce no event.

The sum of model-event counts must equal the total eligible final applied agent-route assignment count. It does not need to equal the environment or agent count from #10442.

## Current source of truth

`InferenceSelection` in `src/lib/inference/selection.ts` records:

- `provider`
- `model`
- `endpointUrl`
- `endpointSource`
- `credentialEnv`
- `preferredInferenceApi`
- Other route-specific fields

A published `SandboxEntry` currently stores one sandbox-level selection along with optional managed-serving provenance. It cannot represent the final applied `spec.sandboxes[].agents[].inference.routes[]` assignments designed by #10904.

The future projector must read the complete persisted applied route state after reference resolution. It must not read authored YAML, count provider or route definitions, or reconstruct multi-agent assignments from the legacy single-route registry shape.

The current registry also cannot distinguish every product-catalog, provider-catalog, direct-input, and legacy model selection. `endpointSource` distinguishes onboarding from `inference-set`; it is not model provenance and must not be used as a substitute.

## Durable model provenance

Add a versioned, closed model-selection provenance record beside each recorded final applied route.

It may record only:

- The approved `model_source` enum.
- The configured `api_family` enum.

It must not record another copy of the model ID, provider ID, endpoint, credential name, UI label, catalog URL, or arbitrary metadata.

Write the provenance atomically with the model route and resolve it for each applied agent-route assignment. If a route changes without valid matching provenance, record `unknown` rather than retaining stale provenance.

Preserve matching provenance through snapshot, restore, clone, recovery, and rebuild operations. Legacy assignments without provenance map to `unknown`.

The telemetry projector must not reconstruct provenance from the model name, provider name, endpoint hostname, namespace, prefix, or other raw value.

## Model-source semantics

Selection workflows must record the source at the time the user or product makes the selection:

- `product_catalog`: selected from a checked-in curated model list or managed serving profile.
- `provider_catalog`: selected from a model catalog fetched from a provider or provider-owned service.
- `local`: selected from detected local runtime inventory or verified local-model state outside a product catalog.
- `custom`: supplied directly through a free-form prompt, command argument, environment override, or custom route.
- `unknown`: legacy, missing, malformed, or otherwise unproven provenance.

A direct override remains `custom` even if its text happens to match a catalog model. A managed serving profile remains `product_catalog` even when the resulting model is served locally.

## Known model keys

Create one checked-in telemetry allowlist that maps exact approved public model IDs to stable product-owned keys.

Rules:

- An exact allowlist match produces its approved key.
- A present model that has no exact match produces `other`.
- A missing model produces `unknown`.
- Matching is local only.
- Do not use prefix, substring, fuzzy, namespace, URL, or endpoint inference.
- Do not dynamically add models returned by a provider catalog.
- Do not transmit allowlist lookup inputs.
- Do not hash unmatched values.
- Changing or adding a key requires the schema review defined by #10435.

A custom selection may produce a known key when it exactly matches the checked-in map. Its `model_source` remains `custom`.

## Provider-profile mapping

Use an exact checked-in mapping from NemoClaw’s durable provider IDs:

| Durable provider ID | Telemetry profile |
| --- | --- |
| `nvidia-prod`, `nvidia-nim`, `nvidia-router` | `nvidia` |
| `openai-api` | `openai` |
| `anthropic-prod` | `anthropic` |
| `gemini-api` | `google-gemini` |
| `openrouter-api` | `openrouter` |
| `hermes-provider` | `hermes` |
| `ollama-local` | `ollama` |
| `vllm-local` | `vllm` |
| `llama-cpp-local` | `llama-cpp` |
| `compatible-endpoint`, `compatible-anthropic-endpoint` | `compatible-endpoint` |

A missing provider maps to `unknown`. Any other present value maps to `custom`. Never transmit the original value.

Do not classify providers from endpoint hostnames, credential names, gateway output, or model namespaces.

## API-family semantics

Persist the configured API family after the normal agent and provider compatibility rules resolve it:

- `openai-completions`
- `openai-responses`
- `anthropic-messages`

Missing, legacy, or unrecognized values map to `unknown`.

This describes the recorded configured route. It does not prove which protocol a live endpoint currently accepts or which protocol a running agent uses.

Do not probe an endpoint or inspect inference traffic for telemetry.

## Observation trigger

Use the complete final applied-state observation from #10442 only after an accepted mutating #10904 child persists agent-route assignments.

Observe after a successful top-level apply or later supported route or lifecycle operation changes the final applied assignment set, model selection, provider profile, provenance, or API family. Observe only after the final state commit and required cleanup.

Do not observe:

- Config export, parsing, validation, planning, or other authored-only YAML work.
- A provider or route definition that is not assigned to an agent.
- Pending route reservations or intermediate route mutations.
- Failed validation or provider setup.
- Failed, cancelled, partially applied, recovery-required, inconclusive, or rolled-back operations.
- A no-op operation.
- Live gateway drift that was not committed to final applied state.
- Read-only status, health, or model-list operations.

A multi-step operation produces at most one final observation batch. An applied-state read or projection failure skips the complete model-event family rather than sending partial counts.

## Privacy boundary

The client must never transmit:

- Raw model IDs, aliases, names, revisions, paths, or hashes.
- Raw provider IDs, aliases, names, or labels.
- Agent, sandbox, route, provider-definition, or deployment names and identifiers.
- Endpoint URLs, hostnames, ports, paths, or endpoint-source fields.
- Credential environment names, credential values, or credential-presence signals.
- Managed-serving preset, recipe, catalog, image, digest, revision, or source identifiers.
- NIM container references.
- Prompts, responses, tools, token counts, request counts, latency, errors, or inference outcomes.
- Live route or model-health results.
- Arbitrary registry fields.

Malicious or private strings must map locally to `other`, `custom`, or `unknown` without appearing in the serialized event, logs, errors, or event names.

## Delivery and reporting

- Apply opt-out and CI or test suppression before reading inference selection state.
- Use the common best-effort delivery path.
- Make no retry and create no durable outbox.
- Do not change onboarding, route mutation, recovery, or lifecycle results when telemetry fails.
- Apply the public minimum-count rule from #10435.
- Describe results as **recorded agent-route assignments**, **recorded model selections**, and **configured provider profiles**.
- Do not describe the counts as model usage, provider traffic, inference requests, unique routes, unique agents, or unique environments.

Recorded state can differ from the live route. Public reports must disclose this limitation.

## Acceptance criteria

- Every eligible final applied agent-route assignment maps to one closed model category combination.
- A route assigned to two agents contributes two; two routes assigned to one agent contribute two.
- Unused provider or route definitions, agents with no route, and sandboxes with no route contribute zero.
- Pending, authored-only, and unapplied route state is excluded.
- Model-event counts reconcile with the eligible final applied assignment count, not the environment or agent count.
- Product-catalog, provider-catalog, custom, local, and legacy paths persist the required explicit provenance.
- A direct override is not reclassified by comparing its value with a catalog.
- Exact approved model IDs map to stable checked-in keys.
- Missing models map to `unknown`; all other unrecognized models map to `other`.
- Approved provider IDs map to their fixed profiles.
- Missing providers map to `unknown`; all other provider values map to `custom`.
- Only the three approved API families appear.
- A route change cannot retain provenance belonging to the previous route.
- Snapshot, restore, clone, recovery, and rebuild preserve provenance only with the matching route.
- No live inference or endpoint probe is performed for telemetry.
- Failed, cancelled, partially applied, recovery-required, inconclusive, and rolled-back operations produce no observation.
- Opt-out, CI, and test suppression prevent projection and network delivery.
- Delivery failure does not change the product result.
- Serialized events contain none of the prohibited fields.
- Public reporting follows the terminology and minimum-count contract from #10435.

## Test plan

Add deterministic tests for:

- Pure final applied route-state-to-event projection and exact assignment-count reconciliation.
- Multi-sandbox, multi-agent, and multi-route applied state.
- The same provider or identical resolved route used by two agents counting twice.
- Unused provider and route definitions, agents without routes, and sandboxes without routes producing no event.
- Pending, authored-only, unapplied, partial, and malformed route state.
- Each model-source producer.
- Product-owned and provider-fetched catalogs.
- Direct prompt, flag, environment, and `inference-set` selections.
- Local runtime and managed serving-profile selections.
- Legacy and malformed provenance.
- Every provider-profile mapping and fallback.
- Every API-family value and fallback.
- Exact model-key matches, missing models, and arbitrary models.
- Private aliases, URLs, control characters, credential-shaped strings, and oversized values.
- Successful final apply and later supported route or lifecycle mutation boundaries, plus failed and nonfinal outcomes.
- Snapshot, clone, restore, recovery, and rebuild provenance handling.
- Opt-out, CI, test suppression, and delivery failure.
- Negative serialization assertions for every prohibited raw field.

Use a local contract server for delivery tests. No live model provider, NVIDIA service, inference request, sandbox, or external API is required.

## Not included

- Model requests, tokens, latency, prompts, responses, tools, or errors.
- Provider traffic or availability.
- Endpoint or credential inventory.
- Declared but unused provider or route-definition inventory.
- Live route, model, or agent health.
- Per-environment or per-install timelines.
- Dynamic provider or model categories.
- Raw or hashed custom values.
- Messaging or general configuration measurement.