langgenius / langgenius/dify

Langfuse: message_trace generations dropped (no cost/latency) for Chatbot/Agent/Completion apps since 1.14.0 — generation created without an observation id

Open Beginner friendly
#37,824 2 comments 2 reactions 0 assignees View on GitHub
project#dify
Dominant language
TypeScript
Stars
156k
Forks
24.6k
Avg merge
22h 9m
Merged PRs (30d)
610

Description

### Self Checks

- [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542).
- [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general).
- [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones.
- [x] I confirm that I am using English to submit this report, otherwise it will be closed.
- [x] 【中文用户 & Non English User】请使用英语提交,否则会被关闭 :)
- [x] Please do not modify this template :) and fill in all the required fields.

### Dify version

1.14.2 (root cause also present on `main`; latest commit to the affected file at time of writing: `f15a8f02e`)

### Cloud or Self Hosted

Self Hosted (Docker / Kubernetes)

### Steps to reproduce

1. Self-host Dify 1.14.x.
2. Create any **non-workflow** app — a **Chatbot** (basic *or* Agent mode) or a **Completion** app.
3. Enable Langfuse tracing for that app (Monitoring → Tracing → Langfuse; public key, secret key, host).
4. Send a chat message so a `message` record is produced.
5. Open the resulting trace in Langfuse.

### ✔️ Expected Behavior

The trace contains an `llm` **generation** observation carrying `model` + token `usage`, so Langfuse computes **cost** and **latency** — the same way Workflow / Advanced-Chat (chatflow) apps do.

### ❌ Actual Behavior

The trace has **only the top-level `message` node** — no child generation. Token counts appear only as plain fields inside the trace **input** payload (`message_tokens`, `answer_tokens`), not as a generation's `usage`. Because there is no generation observation, **Total Cost is blank and Latency shows 0.00s**. Workflow / Advanced-Chat (chatflow) apps are **not** affected.

---

#### Root cause

File: `api/providers/trace/trace-langfuse/src/dify_trace_langfuse/langfuse_trace.py`

`message_trace()` creates the trace (with `id=trace_id`) and then an `llm` generation — but the generation is built **without an `id`**:

```python
langfuse_generation_data = LangfuseGeneration(
name="llm",
trace_id=trace_id,
start_time=trace_info.start_time,
...
# <-- no id= is passed
)
self.add_generation(langfuse_generation_data)
```

`LangfuseGeneration.id` defaults to `None`. `add_generation()` then sends the event through the **low-level ingestion API**, and `filter_none_values` strips the `None` id:

```python
data = filter_none_values(langfuse_generation_data.model_dump()) # drops the None "id"
body = CreateGenerationBody(id=data.get("id"), ...) # id is None
event = IngestionEvent_GenerationCreate(body=body, ...)
self.langfuse_client.api.ingestion.batch(batch=[event])
```

So the generation event reaches Langfuse with **no observation `id`** and is dropped, while the trace (which has its own id) is accepted — hence "trace but no generation, no cost, 0.00s latency".

`workflow_trace()` is **not** affected because it always passes an explicit id:

```python
node_generation_data = LangfuseGeneration(id=node_execution_id, ...)
```

#### Why it regressed

Before 1.14 (e.g. `1.13.3`), `add_generation()` used the **high-level SDK**, which auto-generates a client-side observation id when none is supplied:

```python
self.langfuse_client.generation(**format_generation_data)
```

The trace-provider refactor (#35144) / Langfuse v3-SDK migration switched to the raw ingestion API, which does **not** auto-generate an id — but `message_trace` was never updated to supply one. This broke generation export for every `message_trace` app type (Chatbot, Agent, Completion) from 1.14.0 onward, and it is still present on `main`.

#### Suggested fix

Pass an explicit `id` to the `LangfuseGeneration` created in `message_trace` (and to the other `message`-path observations in this file that currently omit it — `suggested_question_trace`, `moderation_trace`, `dataset_retrieval_trace`, `tool_trace`), e.g.:

```python
langfuse_generation_data = LangfuseGeneration(
id=str(uuid.uuid4()),
name="llm",
trace_id=trace_id,
...
)
```

#### Notes

- The same `MessageTraceInfo` flow is shared by other trace providers (e.g. Opik), so it's worth checking whether they have the same missing-id pattern on their low-level ingestion paths.
- Possibly related: #35421 (incorrect token counts on the same `message_trace` path).

Contributor guide

Open the contributing guide

Research direction

Start in api/providers/trace/trace-langfuse/src/dify_trace_langfuse/langfuse_trace.py, especially message_trace() and the other message-path observation builders named in the issue. Compare their LangfuseGeneration construction with workflow_trace(), then reproduce a non-workflow app trace using Langfuse. Done means message traces contain generation observations with valid ids and Langfuse reports token usage, cost, and latency.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
observability-sre
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
75/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.