deepset-ai / deepset-ai/haystack-core-integrations

Connect the generators to prompts in langfuse

Open
#1,154 14 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

integration:langfuse P3
Dominant language
Python
Stars
203
Forks
332
Avg merge
2d 4h
Merged PRs (30d)
80

Description

Is your feature request related to a problem? Please describe.
Hello! Currently, pipeline runs (equivalent to traces in Langfuse) do not have their corresponding prompts tied with the "generation" blocks. This can be seen in the example from the Haystack Langfuse Integration blog post.
image

Describe the solution you'd like
When prompts exist, they should be added to the tracer file, such as in the following example:

m = meta[0]
try:
    prompt = self._tracer.get_prompt(m.get("prompt_name")) # assumes that in model's output there is an additional parameter "prompt_name"
    if prompt:
        span._span.update(prompt=prompt)
except Exception as e:
    print(f"Prompt not found or error occurred: {e}")
span._span.update(usage=m.get("usage") or None, model=m.get("model"))

This would ensure that prompts are tied to the corresponding generation blocks, giving better traceability.

Describe alternatives you've considered
N/A

Additional context
Desired output:
image

One challenge is that the prompt name is not available in the generator by default. This could complicate the integration, as it may require accessing the prompt metadata manually. In some cases, creating a custom generator that explicitly handles the prompt name and metadata might be necessary for an integration with Langfuse + allowing that custom generator to be traced (see https://github.com/deepset-ai/haystack-core-integrations/issues/1153)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by inspecting the existing Langfuse tracer integration and how generator output metadata is exposed. Determine how prompt names can be obtained, then verify that available prompts appear on the corresponding generation blocks and that usage and model metadata remain attached.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, observability
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.