langfuse / langfuse/codex-observability-plugin

[Feature] Expose developer (system) prompt as a separate observation in traces

Open
#84 0 comments 0 reactions 1 assignee View on GitHub

@milanagm is already working on this.

Since Sep 18, 2026.

[Langfuse] feat-tracing feature
Dominant language
TypeScript
Stars
30
Forks
31
Avg merge
3d 6h
Merged PRs (30d)
13

Description

Description:

Summary

The developer role message in the rollout (the system prompt / instructions) is currently silently dropped by the parser. This is the single largest token consumer in most Codex sessions — often 10k–20k+ tokens, yet it's invisible in Langfuse. I can only infer its size from the gap between input_tokens on each generation and the visible user prompt.

Motivation

When debugging cost or prompt quality, I need to see what is being sent as the system prompt, not just that it's expensive. Common use cases:

  • Verifying that AGENTS.md / <environment_context> content is being injected as expected
  • Comparing system prompt size across repos or configurations
  • Auditing whether instructions are bloating token usage

Current rollout structure (per #47)

Row Role Content
2 developer System prompt / instructions ← dropped
3 user Injected wrapper (<environment_context>, AGENTS.md)
6 user Actual user prompt

Proposal

Export the developer-role message as one of:

  1. A dedicated span/observation (e.g. type span, name system_prompt) attached to the turn, with the full text as its input — or
  2. Part of the generation's input (prefixed, clearly delimited) so it's visible on each LLM call.

Option 1 is preferred — it keeps the generation input clean and lets Langfuse show it as a collapsible sibling.

Implementation sketch

In parse.ts, when iterating rollout rows, capture payload.role === "developer" messages (currently skipped) and emit them as a span before the first generation of the turn:

// pseudo
if (p.role === "developer" && p.type === "message") {
  const sysSpan = startObservation({
    name: "system_prompt",
    type: "span",
    input: p.content,
    // attach to current turn
  });
  sysSpan.end();
}

No additional rollout data is needed, as the content is already parsed, just not exported.

Environment

  • Plugin: tracing@codex-observability-plugin 0.3.0
  • Codex: 26.904.11930
  • Langfuse Cloud EU

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.