OTel: skill invocations are only observable as a metric counter — emit a log event (`codex.skill.injected`) for fleet attribution
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 125k
- Forks
- 19.4k
- PR merge metrics
- PR metrics pending
Description
What variant of Codex are you using?
CLI (the instrumentation lives in codex-rs core, so App / app-server are equally affected)
What feature would you like to see?
Emit skill invocations as an OTel log event (codex.skill.injected), alongside the existing metric counter of the same name — the same pattern codex.tool_decision / codex.tool_result already follow.
Use case
We run Codex across an enterprise fleet (thousands of managed devices) and collect OTel telemetry centrally to measure skill adoption and per-team ROI. Today skill usage is observable only as the codex.skill.injected counter, and a counter is structurally unable to support this, for two independent reasons:
-
The metrics resource carries no identity, by design. The metrics client builds its own resource with only
service.*,envand OS attributes (codex-rs/otel/src/metrics/client.rs,os_resource_attributes), whilehost.nameis attached only forResourceKind::Logs(codex-rs/otel/src/provider.rs,resource_attributes). We verified this in production on 0.148.0: skill-counter datapoints arrive withenv / os / os_version / service.*only, while log events from the very same process do carryhost.name. The code is unchanged onmainas of 3abbf9f (2026-09-14). Counter datapoints therefore cannot be attributed to a device or user, even server-side. -
No conversation linkage. The counter has no conversation/turn identity, so skill usage cannot be joined to
codex.api_request/ token-usage events to measure per-skill cost. Thelog_event!common fields (conversation.id,originator,app.version,terminal.type,model,slug, auth mode) provide exactly this.
Root-cause analysis and a potential approach
The insertion point already exists: emit_explicit_skill_invocations and maybe_emit_implicit_skill_invocation (codex-rs/core/src/skills.rs) fan each invocation out to the metric counter and to analytics_events_client.track_skill_invocations — i.e. OpenAI's internal analytics already receives exactly this event; OTel consumers are the only ones who cannot.
A minimal change (~50 lines + a test), validated against main @ 3abbf9f (rust 1.95.0):
codex-rs/otel/src/events/session_telemetry.rs: addskill_injected(&self, skill_name, status, invoke_type, reasoning_effort, plugin_id: Option<&str>)usinglog_event!(log-only target, mirroringtool_decision). It carries the skill name and the sameinvoke_type/reasoning_effort/plugin_iddimensions the counter has since #40724;model/slugcome from the macro's common fields.codex-rs/core/src/skills.rs: call it right next to the existing counter emission in both the explicit and implicit paths, so the implicit path's per-turn dedup is shared.
Validation: cargo check -p codex-otel -p codex-core clean, the new traced_test-based test skill_injected_records_log_event passes, cargo fmt --check clean.
Validated diff (115 insertions, 3 files, against main @ 3abbf9f)
diff --git a/codex-rs/core/src/skills.rs b/codex-rs/core/src/skills.rs
index ef610de3..01b1b8a3 100644
--- a/codex-rs/core/src/skills.rs
+++ b/codex-rs/core/src/skills.rs
@@ -73,6 +73,13 @@ pub(crate) async fn emit_explicit_skill_invocations(
("reasoning_effort", reasoning_effort.as_str()),
],
);
+ turn_context.session_telemetry.skill_injected(
+ skill.name.as_str(),
+ status,
+ "explicit",
+ reasoning_effort.as_str(),
+ skill.plugin_id.as_deref(),
+ );
}
let injected_host_skill_prompts = turn_context
@@ -195,6 +202,13 @@ pub(crate) async fn maybe_emit_implicit_skill_invocation(
("reasoning_effort", reasoning_effort.as_str()),
],
);
+ turn_context.session_telemetry.skill_injected(
+ skill_name.as_str(),
+ "ok",
+ "implicit",
+ reasoning_effort.as_str(),
+ invocation.plugin_id.as_deref(),
+ );
sess.services
.analytics_events_client
.track_skill_invocations(
diff --git a/codex-rs/core/tests/suite/otel.rs b/codex-rs/core/tests/suite/otel.rs
index 64b8790f..69cab09b 100644
--- a/codex-rs/core/tests/suite/otel.rs
+++ b/codex-rs/core/tests/suite/otel.rs
@@ -1199,6 +1199,70 @@ fn network_policy_decisions_omit_source_and_destination() {
}
}
+#[test]
+#[traced_test]
+fn skill_injected_records_log_event() {
+ let telemetry = SessionTelemetry::new(
+ ThreadId::new(),
+ "gpt-5.5",
+ "gpt-5.5",
+ /*account_id*/ None,
+ /*account_email*/ None,
+ Some(TelemetryAuthMode::ApiKey),
+ "Codex_Desktop".to_string(),
+ /*log_user_prompts*/ false,
+ "tty".to_string(),
+ SessionSource::Cli,
+ );
+
+ telemetry.skill_injected(
+ "linear", "ok", "explicit", /*reasoning_effort*/ "default", /*plugin_id*/ None,
+ );
+ telemetry.skill_injected(
+ "my-plugin:docs-review",
+ "ok",
+ "implicit",
+ /*reasoning_effort*/ "high",
+ /*plugin_id*/ Some("my-plugin"),
+ );
+
+ logs_assert(|lines: &[&str]| {
+ let bare = lines
+ .iter()
+ .find(|line| line.contains("codex.skill.injected") && line.contains("skill=linear"))
+ .ok_or_else(|| "missing skill injection event for bare skill".to_string())?;
+ if !bare.contains("status=ok")
+ || !bare.contains("invoke_type=explicit")
+ || !bare.contains("reasoning_effort=default")
+ {
+ return Err("bare skill event missing status/invoke_type/reasoning_effort".to_string());
+ }
+ if bare.contains("plugin_id=") {
+ return Err("bare skill event unexpectedly included a plugin id".to_string());
+ }
+
+ // The log event carries the skill name as-is, including a plugin
+ // namespace prefix.
+ let namespaced = lines
+ .iter()
+ .find(|line| {
+ line.contains("codex.skill.injected")
+ && line.contains("skill=my-plugin:docs-review")
+ })
+ .ok_or_else(|| "missing skill injection event for plugin skill".to_string())?;
+ if !namespaced.contains("invoke_type=implicit")
+ || !namespaced.contains("reasoning_effort=high")
+ {
+ return Err("plugin skill event missing invoke_type/reasoning_effort".to_string());
+ }
+ if !namespaced.contains("plugin_id=my-plugin") {
+ return Err("plugin skill event missing plugin id".to_string());
+ }
+
+ Ok(())
+ });
+}
+
#[test]
#[traced_test]
fn sandbox_outcome_event_records_outcome() {
diff --git a/codex-rs/otel/src/events/session_telemetry.rs b/codex-rs/otel/src/events/session_telemetry.rs
index 5e8f7ceb..39daa1b1 100644
--- a/codex-rs/otel/src/events/session_telemetry.rs
+++ b/codex-rs/otel/src/events/session_telemetry.rs
@@ -1124,6 +1124,43 @@ impl SessionTelemetry {
}
}
+ /// Records a skill injection as a log event, alongside the existing
+ /// `codex.skill.injected` metric counter emitted by the caller.
+ ///
+ /// The counter alone cannot support per-conversation or per-device
+ /// attribution: the metrics resource intentionally carries no host
+ /// identity and the counter has no conversation linkage. The log event
+ /// complements it with the log resource and the per-session metadata
+ /// (conversation id, originator) attached by `log_event!`.
+ pub fn skill_injected(
+ &self,
+ skill_name: &str,
+ status: &str,
+ invoke_type: &str,
+ reasoning_effort: &str,
+ plugin_id: Option<&str>,
+ ) {
+ match plugin_id {
+ Some(plugin_id) => log_event!(
+ self,
+ event.name = "codex.skill.injected",
+ skill = %skill_name,
+ status = %status,
+ invoke_type = %invoke_type,
+ reasoning_effort = %reasoning_effort,
+ plugin_id = %plugin_id,
+ ),
+ None => log_event!(
+ self,
+ event.name = "codex.skill.injected",
+ skill = %skill_name,
+ status = %status,
+ invoke_type = %invoke_type,
+ reasoning_effort = %reasoning_effort,
+ ),
+ }
+ }
+
pub fn sandbox_outcome(
&self,
tool_name: &str,
Related issues
- #11052 — asked for
skill_nameon tool events; closed NOT_PLANNED for lack of upvotes. This request is the complementary shape: a dedicated event also covers explicit$skillmentions, which create no tool call and are invisible to tool-event attribution. - #41760 — asks for a SkillInvocation event in the local log stream (
logs_2.sqlite) via atracingevent on targetcodex_analytics::skill_invocation. That target is filtered out of OTLP export (is_log_export_targetincodex-rs/otel/src/targets.rsonly passescodex_otel*), so even if merged it would not reach an OTel collector. This request is the exporter-facing counterpart; going throughlog_event!also attachesconversation.id/originator/host.name. - #35650 / #40724 — extend the counter's attributes. #40724 landed
plugin_id/model_slug/reasoning_effort; the remaining gaps (identity, conversation linkage) are what a counter cannot carry at all, which is why a log event is proposed rather than more tags. - #17132 / #39906 — hook events for skill invocations. Hooks are per-user, client-side configuration and cannot serve centrally managed fleet telemetry; the OTel exporter is the managed path.
- #35869 — SkillsExtension-managed plugin skills emit no counter at all. We reproduced this on 0.148.0 (SKILL.md demonstrably injected per the session rollout, yet nothing emitted). A log event at the core fanout keeps parity for that future fix as well.
Additional information
For comparison, Claude Code exposes both a skill_activated log event and a skill.name attribute on API-request events; we use those for the equivalent measurements on that side of our fleet. Happy to share further environment details or test evidence.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with emit_explicit_skill_invocations and maybe_emit_implicit_skill_invocation in codex-rs/core/src/skills.rs, then inspect the existing telemetry methods and log_event! usage in codex-rs/otel/src/events/session_telemetry.rs. Add coverage beside the OTel tests in codex-rs/core/tests/suite/otel.rs and run the named cargo checks, the traced test, and cargo fmt --check; done means both invocation paths produce the documented event fields.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- observability
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100