deepset-ai / deepset-ai/haystack-core-integrations
Connect the generators to prompts in langfuse
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 203
- Forks
- 332
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 80
Description
Is your feature request related to a problem? Please describe.
Hello! Currently, pipeline runs (equivalent to traces in Langfuse) do not have their corresponding prompts tied with the "generation" blocks. This can be seen in the example from the Haystack Langfuse Integration blog post.
Describe the solution you'd like
When prompts exist, they should be added to the tracer file, such as in the following example:
m = meta[0]
try:
prompt = self._tracer.get_prompt(m.get("prompt_name")) # assumes that in model's output there is an additional parameter "prompt_name"
if prompt:
span._span.update(prompt=prompt)
except Exception as e:
print(f"Prompt not found or error occurred: {e}")
span._span.update(usage=m.get("usage") or None, model=m.get("model"))
This would ensure that prompts are tied to the corresponding generation blocks, giving better traceability.
Describe alternatives you've considered
N/A
Additional context
Desired output:
One challenge is that the prompt name is not available in the generator by default. This could complicate the integration, as it may require accessing the prompt metadata manually. In some cases, creating a custom generator that explicitly handles the prompt name and metadata might be necessary for an integration with Langfuse + allowing that custom generator to be traced (see https://github.com/deepset-ai/haystack-core-integrations/issues/1153)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by inspecting the existing Langfuse tracer integration and how generator output metadata is exposed. Determine how prompt names can be obtained, then verify that available prompts appear on the corresponding generation blocks and that usage and model metadata remain attached.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, observability
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100