vllm-project / vllm-project/aibrix

Support otlp_traces_endpoint for trtllm

Open
#2,020 1 comment 0 reactions 0 assignees View on GitHub
area/observation kind/feature priority/important-soon
Dominant language
Go
Stars
5.1k
Forks
694
Avg merge
1d 20h
Merged PRs (30d)
98

Description

### 🚀 Feature Description and Motivation

v1.2.0 and above.

```
# disagg_config.yaml (proxy)
otlp_config:
otlp_traces_endpoint: "http://jaeger:4317"
```

the disagg proxy itself does NOT create spans.
```
Client (traceparent) → Proxy (no span) → CTX Worker (llm_request span) -> → GEN Worker (llm_request span)
```

The two worker spans share the same traceparent from the client, so they appear as siblings under the same trace in Jaeger/Zipkin, but there's no parent span from the proxy showing the orchestration (ctx→gen sequencing, routing decisions, etc.).

some limitations
- No proxy-level spans — routing decisions, ctx/gen orchestration timing not traced
- Worker spans are created at request completion, not request start — they're retroactive spans with corrected timestamps
- No automatic span propagation between proxy→worker HTTP calls — the OpenAIHttpClient doesn't inject traceparent headers into forwarded requests (it passes through the original client's headers if present)
- return_perf_metrics: true is required on workers for spans to have meaningful timing data

### Use Case

observability

### Proposed Solution

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.