[PROPOSAL] RFC: Shift-Left Observability for AI Agents

RFC: Code-Level Runtime Context for AI Agents

Status: Draft v2 for comment. v2 incorporates feedback from an early partner design review.
Author: Anirudha (OpenSearch Observability maintainer)
Target repositories: opensearch-project/observability-stack (primary), opensearch-project/dashboards-observability, opensearch-project/sql, opensearch-project/data-prepper, opensearch-project/opentelemetry-demo
Site: otel.opensearch.org (Observability Stack documentation)
Related: Agentic AI evaluation platform RFC (dashboards-observability#2588), OpenTelemetry GenAI semantic conventions, OpenTelemetry code.*, vcs.*, deployment.*, and cicd.* semantic conventions

Terminology note (v2). v1 used "shift-left observability" as the umbrella. That phrase describes only the pre-merge half of the loop. The umbrella is now Code-Level Runtime Context: the open layer that serves production runtime behavior, addressed by code identity, to AI agents. "Shift left" remains the name for the pre-merge use-case family (blast radius, pre-merge comparison) that consumes this layer, alongside the post-deploy and production use cases that need no CI integration at all.

---

1. Summary

Observability was built for humans reading dashboards after something broke. The primary consumer is now an AI agent that writes, reviews, ships, and repairs code. That agent needs production runtime context, addressed by function, file, commit, and change, at every point in the loop: when a regression appears after a deploy, when a production issue needs a root cause and a fix, and before a change merges.

This RFC proposes that the OpenSearch Observability Stack become the open, OpenTelemetry-native code-level runtime context layer for AI agents. It defines five open contracts and a partner conformance profile:

1. Schema contract. The Runtime Context Schema Profile: telemetry that is code-addressable across spans, logs, metrics, forensic evidence, deployment markers, and function declarations, using OpenTelemetry attributes throughout.
2. Access contract. The Runtime Context Tool Contract: a versioned MCP tool set with defined data sources per tool (metrics and edges when spans are absent), PPL for logs, traces, and long-term metrics, PromQL for operational metrics, and a token-budgeted Context Pack response format.
3. Skills contract. An open registry of Agent Skills (SKILL.md) that teach any coding or SRE agent how to use the layer correctly.
4. Workflow contract. Reference recipes ordered by day-one value: post-deploy verification and issue-to-root-cause first, then blast radius, pre-merge comparison, health reports, cold-code cleanup, and AI token-cost regression, runnable on GitHub Actions, Claude Code, Cursor, Kiro, or any MCP client.
5. Loop contract. Every agent action is itself traced with GenAI semantic conventions and linked to evaluation results, so the layer observes, evaluates, and improves the agents that use it.

Partners that produce runtime data (runtime code sensors, error monitors, APM agents, CI/CD systems, coding agents) integrate through these contracts rather than through bespoke plugins, over OTLP or by keeping data in place behind the tool contract. A "Runtime Context Ready" conformance profile makes the integration testable and listable on otel.opensearch.org.

2. Motivation

2.1 The consumer of observability changed

AI coding agents now write a large share of production code. They plan, edit, test, and review, and their assistance stops at merge. What they lack is what production knows: which functions carry traffic, which entry points regressed after the last deploy, which exceptions are new, which outbound call failed and in what machine state. Static analysis or sampled, hand-picked telemetry cannot answer these questions. Only continuous, code-level runtime data can.

2.2 The market has converged on one pattern

Across function-level runtime sensors, error monitoring platforms, and application performance vendors, the same architecture has emerged in the last twelve months:

Capability	Pattern
Data	Function-level and endpoint-level runtime data from production, with unsampled counts and sampled duration and error evidence, mapped to source file and function
Agent access	A first-party MCP server (local and hosted) plus a query API an agent can call directly
Post-deploy	Release verification, regression attribution to functions, and rollback recommendations from production deltas
Remediation	A pipeline from detected issue to root cause to an opened pull request
Pre-merge	Pull request risk checks that compare the changed functions against live production behavior
Runners	Recipes packaged for GitHub Actions, IDE automations, and scheduled coding-agent tasks
Self-observation	Monitoring of the agents and MCP servers themselves, including LLM calls, tokens, and tool executions

Each implementation is proprietary: proprietary schema, proprietary query language, proprietary MCP tools. A team that adopts one cannot move its agent workflows to another, and a team that runs OpenSearch for observability cannot use its own data in any of them.

2.3 OpenSearch's position

The Observability Stack already has the foundation: OTLP ingestion, OpenTelemetry semantic conventions as the canonical schema (including GenAI conventions for agent traces), PPL with pattern mining and anomaly detection built into the query pipeline, Prometheus-compatible metrics, and a Claude Code plugin with an MCP server and skills scaffold in claude-code-observability-plugin/. What is missing is the set of contracts that turn this into something an agent, and a partner, can build on without asking a maintainer.

The opportunity is to be the open substrate for code-level runtime context: the place where runtime behavior from any source lands in a standard shape and is served to any agent through standard interfaces.

3. Goals and non-goals

Goals

• An AI coding agent working in any repository can obtain production runtime context for the code it is changing in one tool call, with a bounded token cost, whether the underlying source emits spans, metrics, or both.
• A production regression after a deploy can be detected, attributed to functions and entry points, and given a verdict, and a production issue can flow to a root cause and a remediation pull request, using OpenSearch, open-source components, and any conformant runtime data source. Every agent step is traced and evaluable.
• A pull request can be gated on production evidence (traffic, latency, errors, token cost).
• A third party can integrate a runtime data source or an agent workflow by conforming to published contracts and passing a conformance suite.

Non-goals

• Building a first-party function-level runtime sensor. Function-level data enters through OTLP from partners and from OpenTelemetry instrumentation, or stays in place behind a conformant Context surface; partners are not required to ingest into OpenSearch to participate.
• Replacing OpenSearch Dashboards or the Investigations experience. Those remain the human surfaces; this RFC defines the agent surfaces.
• Prescribing a single coding agent or CI system.

4. Proposal overview

Coding / SRE agents (Claude Code, Cursor, Kiro, Codex, custom)
        |  MCP tools + PPL + PromQL + Skills
        v
+------------------------------------------------------------+
| Runtime Context Access Layer (observability-stack MCP srv)  |
|  get_runtime_context | compare_windows | search_issues      |
|  get_deployments | get_evidence | blast_radius              |
|  query_ppl | query_promql | list_skills | record_agent_action|
+------------------------------------------------------------+
        |
        v
+------------------------------------------------------------+
| OpenSearch + Prometheus (OTel semconv schema profile)       |
|  spans (code.*, vcs.*, deployment.*, gen_ai.*)              |
|  logs (exception.*, error.fingerprint, trace context)       |
|  function metrics (code.function.*, flows, callers)         |
|  runtime & machine metrics (process.*, event loop)          |
|  edges | forensic evidence | function declarations          |
|  deployment markers | agent-actions | evaluation results    |
+------------------------------------------------------------+
        ^
        |  OTLP (schema profile) | Context surface (data in
        |  place) | workflow recipes
Runtime data producers: OTel SDKs, runtime code sensors, error
monitors, APM agents, CI/CD systems, coding agents (self-trace)

Ruling infrastructure in or out is step one of deployment impact analysis, so runtime and machine metrics sit in the profile alongside function metrics rather than outside it.

5. Detailed design

5.1 Schema contract: the Runtime Context Schema Profile

The stack already indexes OpenTelemetry attributes as-is. This section defines the minimum attribute set that makes telemetry addressable from a repository. All names are OpenTelemetry semantic conventions unless marked proposed.

Attribute	Signal	Purpose	Status
service.name, service.version, service.instance.id	all	Service identity and release	Stable
deployment.environment.name	all	Separate production from preview and staging cohorts	Stable
cloud.region, k8s.cluster.name	resource	Locate regressions by placement; recommended for multi-region and multi-cluster fleets	Development
code.function.name, code.file.path	spans, logs, metrics	Code unit identity. code.function.name is fully qualified (the former code.namespace is folded into it)	Stable
code.line.number, code.column.number	spans, logs only	Precise location for forensics. Excluded from metric series: line numbers re-key series on every edit above the function	Stable
code.flow.name, code.flow.type	metrics, spans	The entry point (flow) a function executed within: HTTP route, queue consumer, scheduled job, GraphQL operation. Without it an agent cannot answer which function is responsible for an endpoint's slowdown	Proposed; alternative is reusing http.route / messaging.destination.name / rpc.method, see open questions
code.function.caller.name	metrics, spans	Call-graph edge; fully qualified, optional in v0. The edges index (5.2) is the authoritative graph	Proposed; upstream via OTel semconv SIG
code.stacktrace, exception.type, exception.message, exception.stacktrace	span events, logs	Error forensics	Stable
error.fingerprint	logs, error metrics, evidence	Producer-supplied issue identity that survives the boundary between a partner source and the stack	Proposed
vcs.repository.url.full, vcs.ref.head.revision, vcs.ref.head.name	resource	Tie telemetry to a commit and branch	Development
vcs.change.id, vcs.change.state	resource	Tag preview or canary telemetry with the pull request under test	Development
deployment.id, deployment.name, deployment.status	deployment markers	Before/after comparison windows. deployment.status=detected (proposed value) covers sensor-detected deployments inferred from changed source hashes, with vcs.* optional in that case	Development; detected proposed
cicd.pipeline.name, cicd.pipeline.run.id	deployment markers, agent actions	Link gates and runs to CI	Development
gen_ai.operation.name, gen_ai.agent.name, gen_ai.tool.name, gen_ai.usage.input_tokens, gen_ai.usage.output_tokens, gen_ai.request.model	spans	Observe AI applications and the runtime-context agents themselves	Development
gen_ai.evaluation.result event, test.case.id, test.suite.run.id	spans, evaluation records	Link agent output to evaluation outcomes	Development / proposed in OTel GenAI WG

Function-level metrics. Runtime sensors export function-level data as OpenTelemetry metrics rather than as a proprietary table. Proposed instrument set (experimental, to be confirmed with the OTel semantic conventions SIG):

Instrument	Type	Unit	Semantics
code.function.calls	Counter	{call}	Authoritative invocation count. Sensors keep exact counts
code.function.call.duration	Histogram	s	Duration distribution. Observations MAY be sampled, so its count MAY be lower than code.function.calls and MUST NOT be used as a traffic measure
code.function.errors	Counter	{error}	Errored invocations, with error.type, optional error.fingerprint, and a caught/escaped boolean (attribute name pending upstream)

Series identity is code.function.name plus code.file.path. Dimensions on all three: code.flow.name, code.flow.type (Recommended), code.function.caller.name (Recommended, optional in v0, with a mandatory disable switch). To keep Prometheus series lean, sensors and the Collector MAY keep only function identity in Prometheus and route caller and flow dimensions to OpenSearch long-term storage; sensors already aggregate in time and throttle hot functions, and the Collector can reduce dimensions in transit. Custom low-cardinality tags MAY be promoted to Prometheus labels through the existing promote_resource_attributes path.

Runtime and machine metrics. Recommended alongside function metrics: process.cpu.utilization, process.memory.usage, process uptime, and runtime-specific signals such as Node.js event loop utilization, per instance. Deployment impact analysis begins by ruling infrastructure in or out.

Function declarations. Cold-code detection needs an inventory: a never-called function has no metric series. A code.function.declared event (proposed) carries code.function.name, code.file.path, service.name, and vcs.ref.head.revision, emitted by sensors or build tooling at startup or deploy, routed to code-declarations-otel-v1-*.

Deployment markers. A deployment is recorded as an OTel log record (event name deployment) carrying deployment.*, service.*, and where available vcs.* and cicd.*. Two producers are conformant: CI emitters (a GitHub Action and a curl one-liner are provided; vcs.* required) and runtime sensors that detect deployments from changed source hashes (deployment.status=detected; vcs.* optional). Data Prepper routes these to deployments-otel-v1-*.

Pipeline changes. The Collector transform already flattens code.function.name. The profile extends this to the full code.*, vcs.*, and deployment.* set. Data Prepper index templates add keyword mappings for attributes.code.function.name, attributes.code.file.path, attributes.code.flow.name, attributes.error.fingerprint, resource.vcs.change.id, resource.vcs.ref.head.revision, and resource.deployment.id. Field-level security policies cover attributes.gen_ai.*.arguments, body, exception.message, and evidence payloads for PII control.

5.2 Access contract: the Runtime Context Tool Contract

The MCP server in claude-code-observability-plugin/ becomes the reference implementation of the Runtime Context Tool Contract v0. The contract is small, read-mostly, and stable.

Tool	Purpose	Inputs	Output
get_runtime_context	Production behavior for the code an agent is touching	repo, ref, one of paths[] / functions[] / flows[] / endpoints[] / services[], environment, window, budget_tokens	Context Pack
compare_windows	Baseline vs candidate deltas for a selector	selector (functions, flows, services, endpoints, or gen_ai.agent.name), baseline (window or cohort), candidate (window, deployment.id, or vcs.change.id)	Deltas for calls, errors, p50/p90/p99/p100, tokens, cost, with sample sizes
search_issues	New and regressed error identities	services[] / flows[] / functions[], since, environment	Issues keyed by producer error.fingerprint where supplied, else clustered patterns from PPL patterns, with counts, first seen, last seen, source function, example evidence id
get_deployments	Deployment markers	service, since	Marker list with deployment.id, status (including detected), revision and PR when known
get_evidence	Forensics for a specific failure or slow execution	one of trace_id, error.fingerprint, evidence_id	Typed evidence: a span tree, a forensic execution sample (functions in order, exceptions, machine state), or a correlated log set, depending on the producer
blast_radius	Rank changed functions and endpoints by production exposure, with callers and downstream services	diff or changed_files[], environment, window	Ranked list with calls, p90, error rate, callers, flow and service-map neighbors
query_ppl	Escape hatch for logs, traces, edges, long-term metrics	query, max_rows, max_bytes	Rows plus the executed query for provenance
query_promql	Escape hatch for operational metrics	query, start, end, step	Series
list_skills, get_skill	Discover and load skills from the registry	name	SKILL.md content
record_agent_action	Closed loop: write an agent action as an OTel event	action, target (fingerprint, function, PR), gen_ai.* context	Event id

Data sources per tool. The contract is explicit about where each answer comes from, because a conformant source may emit metrics and forensic evidence with no spans at all:

Tool	Spans	Function metrics	Edges	Evidence	Declarations	Markers
get_runtime_context	optional	primary	callers, flows	top exceptions	cold-code	last deploy
compare_windows	optional	primary	no	no	no	window bounds
search_issues	optional	error counters	no	fingerprints, samples	no	since-deploy filter
get_evidence	when present	no	no	primary	no	no
blast_radius	optional	primary	primary	no	optional	no

blast_radius and compare_windows MUST return correct results from metrics and edges alone; span-based enrichment (exemplars, trace deep links) is additive.

Transport and auth. Local stdio for IDE and terminal agents; streamable HTTP for hosted use. Authentication uses the OpenSearch security plugin: a runtime_context_reader role with read access to otel-v1-apm-span-*, otel-v1-apm-log-*, ss4o_metrics-*, edges-otel-v1-*, evidence-otel-v1-*, code-declarations-otel-v1-*, and deployments-otel-v1-* (index naming for sensor-emitted data is an open question), and a runtime_context_agent_writer role with write access only to agent-actions-otel-v1-*. Managed deployments use SigV4 or OIDC.

Response budgets. Every tool accepts max_rows (default 200) and max_bytes (default 256 KB). get_runtime_context accepts budget_tokens and truncates by priority: errors, then regressions, then traffic, then trend.

Reference queries. PPL is the default for logs, traces, edges, and long-term metrics; PromQL for operational metrics. Queries are labeled by required source.

Post-deploy regression attribution from function metrics (metrics-only source):

source = ss4o_metrics-*
| where name = 'code.function.call.duration'
  AND `attributes.code.flow.name` = 'POST /checkout'
| eval cohort = if(`@timestamp` > '2026-09-01 14:00:00', 'after', 'before')
| stats percentile(value, 90) as p90_s, count() as observations
        by cohort, `attributes.code.function.name`
| sort - p90_s

Traffic and error rate for changed functions (metrics-only source):

source = ss4o_metrics-*
| where name IN ('code.function.calls', 'code.function.errors')
  AND `attributes.code.function.name` IN ('checkout.applyDiscount', 'checkout.computeTax')
| stats sum(value) as total by name, `attributes.code.function.name`, `attributes.code.flow.name`

Callers of a changed function. Span parentage is used when function spans exist:

source = otel-v1-apm-span-*
| where `attributes.code.function.name` = 'checkout.applyDiscount'
| join left=c right=p ON c.parentSpanId = p.spanId otel-v1-apm-span-*
| stats count() as calls by p.serviceName, p.name
| sort - calls | head 20

Otherwise callers resolve from the edges index. Edge documents are {from, to, kind, calls, window_start, window_end, service.name} with two kinds: function_caller (function to caller within a service) and flow_service (flow to parent or root service across services):

source = edges-otel-v1-*
| where kind = 'function_caller' AND to = 'checkout.applyDiscount'
| stats sum(calls) as calls by from
| sort - calls | head 20

New error identities since a deployment. Producer fingerprints are preferred; pattern clustering is the fallback:

source = otel-v1-apm-log-*
| where severityNumber >= 17 AND `@timestamp` > '2026-09-01 14:00:00'
| eval issue = if(isnotnull(`attributes.error.fingerprint`), `attributes.error.fingerprint`, 'unfingerprinted')
| stats count() as occurrences, earliest(`@timestamp`) as first_seen by issue, serviceName
| sort - occurrences | head 20

Candidate (pull request preview) vs production baseline, when spans and cohort tagging are present:

source = otel-v1-apm-span-*
| where `resource.vcs.change.id` = '4812'
   OR `resource.deployment.environment.name` = 'production'
| eval cohort = if(`resource.vcs.change.id` = '4812', 'candidate', 'baseline'),
       duration_ms = durationInNanos / 1000000
| stats count() as calls,
        percentile(duration_ms, 90) as p90_ms,
        sum(if(`status.code` = 2, 1, 0)) as errors
        by cohort, `attributes.code.function.name`

Token-cost regression for an AI application change:

source = otel-v1-apm-span-*
| where `attributes.gen_ai.operation.name` IN ('chat', 'invoke_agent')
| eval cohort = if(`resource.vcs.change.id` = '4812', 'candidate', 'baseline'),
       tokens = `attributes.gen_ai.usage.input_tokens` + `attributes.gen_ai.usage.output_tokens`
| stats avg(tokens) as avg_tokens, count() as calls
        by cohort, `attributes.gen_ai.agent.name`

Function metrics in PromQL:

sum by (code_function_name) (rate(code_function_calls_total{service_name="checkout"}[5m]))

Context Pack format (v0). A compact JSON document designed to be pasted into a model context:

{
  "schema": "opensearch.runtime-context.context-pack/v0",
  "scope": {"repo": "acme/checkout", "ref": "feat/discounts", "environment": "production", "window": "7d"},
  "budget": {"tokens": 4000, "truncated": false},
  "functions": [{
    "code.function.name": "checkout.applyDiscount",
    "code.file.path": "src/checkout/discount.ts",
    "runtime": "nodejs",
    "calls": 4152034, "calls_per_min": 412.3,
    "p50_ms": 12, "p90_ms": 88, "p99_ms": 410, "p100_ms": 2210,
    "error_rate": 0.012,
    "first_seen": "2026-03-11T08:00Z", "last_seen": "2026-09-01T13:59Z",
    "flows": [{"code.flow.name": "POST /checkout", "code.flow.type": "http", "share": 0.94}],
    "top_exceptions": [{"fingerprint": "f3a1", "exception.type": "TypeError", "caught": false, "count": 91, "first_seen": "2026-08-29T02:14Z", "evidence_id": "ev-88121"}],
    "callers": ["api.postCheckout"], "callees": ["pricing.lookupRule", "tax.compute"],
    "last_deployment": {"deployment.id": "d-2291", "deployment.status": "detected", "at": "2026-08-31T19:02Z"},
    "machine_at_last_incident": {"cpu": 0.41, "memory": 0.78, "event_loop": 0.93},
    "trend_vs_prior_window": {"calls": "+8%", "p90_ms": "+35%", "error_rate": "flat"}
  }],
  "services": [{"service.name": "checkout", "rate_rps": 61.2, "error_rate": 0.004, "p90_ms": 140, "slo_burn_rate_1h": 0.6}],
  "evidence": [{"tool": "query_ppl", "query": "source = ss4o_metrics-* | where ..."}],
  "links": [{"label": "Flow view", "url": "https://dashboards.example/app/observability#/..."}]
}

The evidence array carries the exact queries used, so an agent (or a reviewer) can re-run and verify rather than trust a summary.

Agent-readable documentation. otel.opensearch.org publishes llms.txt at the root, a Markdown rendering of every page, and an OpenAPI description of the PPL, PromQL, and MCP endpoints.

5.3 Skills contract: the open skills registry

Agent Skills (SKILL.md with frontmatter name, description, allowed-tools) are the portable way to teach an agent how to use the layer. The registry lives in observability-stack/skills/ with a generated index.json served from otel.opensearch.org so any MCP client can discover skills through list_skills.

Skill	Status	Content
ppl-reference	Exists in plugin	Full PPL command and function reference
logs	Exists in plugin	Severity filtering, trace correlation, error patterns, volume analysis
apm-red	Exists in plugin	Rate, errors, duration in PromQL and PPL
correlation	Exists in plugin	Cross-signal correlation on OTel fields and exemplars
slo-sli	Exists in plugin	Recording rules, error budgets, burn-rate alerts
runtime-context/post-deploy-verify	New, v0	Deployment markers (CI-emitted and detected), comparison windows, sample-size gates, rollback recommendation
runtime-context/issue-root-cause	New, v0	Issue to evidence to root cause to PR, with required evidence and self-tracing
runtime-context/blast-radius	New	Static analysis of a diff crossed with production exposure. Needs no candidate traffic: it answers "how exposed is this change" before any deploy
runtime-context/pre-merge-comparison	New	Behavioral delta between a candidate cohort (tagged with vcs.change.id) and the production baseline. Requires candidate traffic, so it depends on a preview or canary environment
runtime-context/performance-bottlenecks	New	Attributing flow-level latency to functions from metrics and edges
runtime-context/ai-cost-regression	New	Token, latency, and evaluation deltas for AI application changes
agent-observability	New	Instrumenting the runtime-context agent itself with GenAI conventions

Partners contribute skills under skills/partners/<name>/ after passing conformance. Skills are Apache 2.0.

5.4 Workflow contract: reference recipes and runners

Each recipe is a documented, testable flow with declared inputs, tools, verdict format, and thresholds. Verdicts gate on minimum sample size in addition to elapsed time, and a latency-only regression with no new fingerprint can still yield a verdict. Ordered by rollout wave:

Recipe	Wave	Trigger	Tools	Output
Post-deploy verification	v0	Deployment marker received, CI-emitted or sensor-detected	get_deployments, compare_windows, search_issues	Verdict when minimum sample size and elapsed-time gates (defaults: 15 and 60 minutes) are both met; rollback recommendation with evidence
Issue to root cause and PR	v0	New error.fingerprint in production	search_issues, get_evidence, get_runtime_context, record_agent_action	Draft PR with root cause, evidence links, and test
Weekly service health	v0	Schedule	get_runtime_context, query_promql	Report of regressions, SLO burn, top new issues, by service
PR blast radius	v1	pull_request opened or updated	blast_radius, get_runtime_context	PR comment with ranked exposure and a risk label. Needs diff-to-function mapping and the edges index
Cold-code cleanup	v1	Schedule	query_ppl over declarations and calls	PR proposing removal, with production evidence
AI token-cost regression	v1	Prompt, model, or tool change in an AI application	compare_windows on gen_ai.*	Pass/fail on tokens per operation, latency, evaluation score
Pre-merge behavioral comparison	v2	Preview environment deployed with vcs.change.id	compare_windows	Pass/fail check against thresholds. Needs cohort tagging and a preview or canary environment

Reference implementations.

• Claude Code plugin commands: /verify-deploy, /root-cause <fingerprint>, /health-report, then /blast-radius and /compare-pr.
• opensearch-project/runtime-context-action: a composite GitHub Action wrapping the MCP server in headless mode, with verify, blast-radius, and compare subcommands and a PR comment template (v1).
• Cursor and Kiro: equivalent rules and hooks in runners/.
• A deployment-marker script and GitHub Action step that emit the marker event on every deploy, complementing sensor-detected markers.

5.5 Loop contract: observe, evaluate, improve the agents

Runtime-context agents are AI agents. They are instrumented with the same GenAI conventions the layer already serves:

• Every recipe run is a trace: gen_ai.operation.name = invoke_agent, gen_ai.agent.name = <recipe>, tool calls as execute_tool spans with gen_ai.tool.name, token usage on every model call.
• record_agent_action writes the business outcome (PR opened, verdict issued, rollback recommended) with vcs.change.id, target code.function.name, and the originating error.fingerprint.
• When post-deploy verification passes or fails for a remediation PR, an evaluation record is written with gen_ai.evaluation.result linked to the remediation trace.

This gives the Investigations experience the data to answer "which agent fixed which fingerprint, at what token cost, and did it hold", and gives the evaluation platform (RFC 2588) online signals for remediation quality. The MCP server monitors itself the same way.

5.6 Partner integration model and conformance

Three integration surfaces, each defined by the contracts above. A partner may be listed at any single surface; the Context surface alone is a complete integration.

Surface	Who	What they implement
Ingest	Runtime code sensors, error monitors, APM agents, CI/CD systems	Export OTLP conforming to the Runtime Context Schema Profile: function metrics with flows and callers, error.fingerprint, deployment markers (vcs.* required for CI emitters, optional for sensors using detected), declarations for cold-code, PII scrubbed at source
Context	Vendors that keep data in place	Expose an MCP server implementing the minimum tool set: get_runtime_context, get_deployments, compare_windows, search_issues, and get_evidence, with the Context Pack format. blast_radius qualifies for the second conformance level. Data stays in place; only context moves
Workflow	Coding agents, IDEs, CI platforms, automation vendors	Ship recipes and runners; emit agent traces with GenAI conventions; call record_agent_action

"Runtime Context Ready" conformance profile. A test suite in observability-stack/conformance/ verifies:

1. Required attributes present and correctly typed on a sample workload, including metrics-only workloads.
2. Deployment markers emitted and indexed, both producer types.
3. Tool contract responses validate against the JSON schemas, including budget enforcement, evidence, and correct answers from metrics and edges with no spans present.
4. Function-level metrics resolve in PromQL and PPL with expected dimensions; duration sampling semantics honored (calls used for traffic).
5. Agent self-traces present for at least one recipe.
6. PII policy: applies to evidence payloads and log bodies. Metrics-only exports carry identifiers rather than payloads, and function names and paths remain internal-sensitive.

Conformant integrations are listed on otel.opensearch.org with their surface level. The listing is a project artifact, not a commercial program; commercial partnership terms are outside this RFC.

5.7 Security and governance

• Read-only by default. Only record_agent_action writes, and only to agent-actions-otel-v1-*.
• Least privilege via OpenSearch security roles (runtime_context_reader, runtime_context_agent_writer) and field-level security. Index patterns and redaction rules are documented per role.
• Every MCP call is audited twice: in the OpenSearch audit log and as a span from the MCP server itself.
• Response budgets and query timeouts are enforced server-side. query_ppl rejects delete-class commands and enforces a wall-clock limit.
• Data residency: the Context surface lets partners and customers keep raw data in place, which enterprise VoC identifies as a hard requirement at petabyte scale.

5.8 Documentation and site

otel.opensearch.org adds a top-level Code-Level Runtime Context section:

1. Overview and the five contracts
2. Get started in five minutes: Claude Code, Cursor, Kiro, GitHub Actions
3. Runtime Context Schema Profile reference
4. Runtime Context Tool Contract reference (with JSON schemas and per-tool data sources)
5. Skills registry
6. Use cases: Post-Deploy Verification, Issue to Root Cause, Shift Left (blast radius and pre-merge), Cost Regression
7. Partner integration and conformance
8. llms.txt and per-page Markdown for agents

6. Reference journeys

A production issue becomes a PR and is verified. This is the primary journey observed in the field today. A new fingerprint appears in production. The issue-root-cause recipe pulls the evidence: the forensic sample showing checkout.applyDiscount raised a TypeError after a failed outbound call to the pricing service, the functions executed in order before the failure, and machine state at the time (event loop utilization at 93 percent). get_deployments ties the fingerprint's first occurrence to a detected deployment two hours earlier. The agent opens a draft PR with the root cause and the evidence queries, and records the action. After the fix deploys, post-deploy verification waits for its sample-size and time gates, confirms the fingerprint stopped and latency held, and writes an evaluation result linked to the remediation trace.

A deploy regresses latency with no new errors. A sensor-detected marker lands at 14:00. At the sample-size gate, compare_windows on flow POST /checkout shows p90 up 35 percent with no new fingerprints. Attribution over function metrics points at pricing.lookupRule; machine metrics rule out infrastructure. The recipe issues a fail verdict with a rollback recommendation and the two queries behind it.

A developer changes a hot function. In Claude Code, the agent calls get_runtime_context with the changed paths. The Context Pack shows checkout.applyDiscount at 412 calls per minute, dominant flow POST /checkout, p90 up 35 percent since the last deploy, and an uncaught TypeError first seen two days ago. The agent adds a guard, writes a test reproducing the failure shape from the evidence, and opens a PR citing the queries. In CI, blast radius labels the PR high because it touches two functions on the checkout flow.

7. Alternatives considered

Alternative	Decision
Proprietary REST API for agents instead of MCP	Rejected. MCP is the transport coding agents already speak; REST remains available underneath
SQL as the primary agent query language	PPL is primary for logs, traces, edges, and long-term metrics because patterns, ml, join, and span compose in one pipeline. SQL is supported for partners that expose SQL sources
Spans as the only data model	Rejected. Every-invocation observation is only viable as client-aggregated metrics; the contract works from metrics, edges, and evidence with spans as enrichment
Build a first-party function-level sensor	Rejected for this RFC. The project owns the schema and the access layer; data arrives via OTLP or stays in place behind the Context surface
One plugin per coding agent	Replaced by MCP plus skills. Runners are thin
Dashboard-first agentic UX only	Complementary. Investigations serves humans; this RFC serves agents. Both read the same data
Vendor-specific attribute names	Rejected. Everything upstreams to OpenTelemetry semantic conventions; proposed attributes are marked and tracked
"Shift-left" as the umbrella term	Revised in v2. Shift-left names the pre-merge use cases; the layer itself is code-level runtime context, which also serves post-deploy and production loops

8. Rollout

Phase	Window	Deliverables
0	Sep to Oct 2026	RFC accepted; Schema Profile doc including function metrics, flows, edges, declarations, and both deployment-marker producers; Collector and Data Prepper template changes; Tool Contract v0 (including get_evidence and per-tool data sources) in the plugin MCP server; v0 skills; llms.txt on the site
1	Nov to Dec 2026	Post-deploy verification and issue-to-root-cause recipes with sample-size verdict gates; deployment-marker emitter alongside detected markers; compare_windows over metrics; first external integration passes conformance at the Context surface
2	Q1 2027	Blast radius with diff-to-function mapping and the edges index; runtime-context-action for GitHub; cold-code cleanup over declarations; AI token-cost regression gate; Investigations reads agent actions
3	Q2 2027	Pre-merge behavioral comparison with cohort tagging; conformance program and listings on otel.opensearch.org; federated context sources; upstream proposals for code.function.* metrics, flow attribution, and code.function.caller.name submitted to OTel

9. Success metrics

• Weekly active MCP clients against the layer, and Context Packs served per week.
• Percentage of deployments receiving an automated verdict, and median time from deployment marker to verdict.
• Median time from fingerprint first seen to remediation PR opened; percentage of remediation PRs merged; percentage that hold at 7 days.
• Pull requests annotated or gated per week once blast radius ships; false-positive rate of high labels as judged by reviewers.
• Tokens consumed per successful root cause, tracked toward the existing 10x reduction target.
• Number of conformant integrations by surface level; skills installed from the registry.

10. Open questions

1. Index naming for sensor-emitted data: whether function metrics, evidence, edges, and declarations belong under the otel-v1-apm-* family, the ss4o_* family, or a new runtime-context family. The reader role definition follows the answer.
2. Flow attribution upstream: code.flow.name plus code.flow.type as new attributes, or reuse of http.route, messaging.destination.name, and rpc.method per flow type.
3. Attribute naming for caught vs escaped errors on code.function.errors, coordinated with the status of exception.escaped upstream.
4. Whether deployment.status=detected is upstreamable as an enum value or remains a profile-level extension.
5. Enforcement of fully qualified code.function.caller.name values, and whether the edges index should carry ids to disambiguate short names.
6. Retention and rollup policy for edges and declarations indices.
7. Hosted MCP authentication for managed deployments: SigV4 passthrough vs OIDC device flow.
8. Threshold defaults for verdicts, and whether they should derive from SLOs already defined in the stack.
9. Governance of partner-contributed skills and recipes: review process, versioning, and deprecation.

11. Request for comment

Feedback is requested on the Tool Contract v0 surface (particularly get_evidence and the per-tool data sources), the Runtime Context Schema Profile including the function-metric semantics and flow attribution, the edges and declarations designs, the Context Pack format, and the conformance checklist. Prototype PRs for Phase 0 will follow acceptance.

Start with the RFC overview and detailed design, especially the five proposed contracts and the Runtime Context Access Layer. Review the target repositories listed in the proposal before deciding where implementation belongs. Done would require an agreed, scoped implementation plan for the contracts and conformance profile; the RFC does not yet define a newcomer-sized change.