Proposal: Add a custom attribute to the entry span for LLM trace identification

Add GenAI Trace Count Metric for Mixed Application Scenarios

Background

In modern AI applications, LLM components are often integrated with traditional infrastructure components (Redis, databases, web services, message queues, etc.). This creates mixed tracing scenarios where a single trace may contain both:

1. Pure GenAI traces: Traces consisting entirely of LLM-related spans
2. Mixed traces: Traces containing both traditional infrastructure spans and GenAI spans

Our team is leveraging OpenLLMetry and opentelemetry-python-contrib to build a new LLM monitoring platform. This means we'll be building a combined agent by installing instrumentation packages from openllmetry on top of the opentelemetry-python-contrib agent. One of the key challenges is we need to expose a new metric representing the count of traces that contain LLM spans(like fastapi -> langchain(mixed) or langchain -> chromadb(pure LLM)).

Current Challenge

Currently, there's no straightforward way to distinguish and count traces that involve GenAI operations. This makes it difficult to:

• Monitor GenAI adoption and usage patterns in mixed applications
• Understand the proportion of requests that involve AI processing vs traditional processing
• Set up proper alerting and monitoring for GenAI-specific operations

Use Case Example

Consider a typical e-commerce application that has integrated AI features:

HTTP Request → Web Service → Database → LLM (Product Recommendations) → Response
     ↓              ↓           ↓              ↓
Traditional    Traditional  Traditional    GenAI Span
   Span          Span        Span

In this scenario, we need to identify that this entire trace involved GenAI processing, even though it contains traditional infrastructure spans.

Our Goal: Counter Metrics Support of Mixed Instrumentation

This approach correctly handles mixed instrumentation scenarios:

Scenario	genai_trace_count
Web(root) → LLM(child)	1
LLM(root)	1
Web(root) → LLM1 + LLM2	1

Proposed Solution

New Metric: genai_trace_count

Add a new counter metric that tracks the number of completed traces that contain GenAI operations.

Metric Specification:

• Name: genai_trace_count
• Type: Counter
• Description: "Number of completed traces containing GenAI operations"
• Unit: "1"
• Labels: Could include trace status (success/error), root service, etc.

Implementation Approach

1. Trace-Level Counting Strategy

We count each unique trace that contains GenAI operations:

def on_end(self, span: ReadableSpan) -> None:
    trace_id = span.context.trace_id
    
    if trace_id not in self._counted_traces:
        self._counted_traces[trace_id] = timestamp
        self._genai_trace_counter.add(1)

2. Core Data Structure

Use OrderedDict to store trace IDs with timestamps for efficient cleanup:

_counted_traces: OrderedDict[int, float] = OrderedDict()
#                            ↑     ↑
#                       trace_id  timestamp

3. Deduplication Logic

Problem: Multiple GenAI spans in the same trace would cause over-counting
Solution: Track trace IDs instead of individual spans

Example:

Flask Request (trace_id: 123)
├── LangChain Call (trace_id: 123) ← First GenAI span: COUNT +1
└── OpenAI Call (trace_id: 123) ← Same trace: SKIP

4. Memory Management

Implement automatic cleanup to prevent memory leaks:

• Time-based cleanup: Remove traces older than TTL (default: 1 hour)
• Size-based cleanup: Remove oldest traces when exceeding limit (default: 50K)
• Periodic cleanup: Run cleanup every 5 minutes or when limit exceeded

Integration Points

1. TracerWrapper: Add the processor to the tracer provider
2. Metrics Configuration: Include the new metric in metric views
3. Initialization: Add configuration parameter to Traceloop.init()

Backward Compatibility

• This is a purely additive feature
• No breaking changes to existing APIs
• Can be disabled via configuration if not needed
• Does not affect existing span processing or metrics

The proposed approach provides accurate, real-time counting with minimal overhead.

Since mixed instrumentation traces are infrequent and to mitigate potential performance overhead, this metric will be disabled by default and activated using an environment variable.

@nirga If you approve this proposal, I'll submit a PR to implement this functionality. The implementation shouldn't be overly complex.

Start by reviewing the proposed on_end trace-counting strategy and the TracerWrapper, metric views, and Traceloop.init() integration points named in the issue. Define how configuration and cleanup should work before implementing the opt-in genai_trace_count metric. Done means completed traces containing GenAI operations are counted once, mixed traces are supported, and the feature remains disabled by default.