deepset-ai / deepset-ai/haystack-core-integrations
Langfuse makes haystack slow
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 203
- Forks
- 332
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 80
Description
Describe the bug
Langfuse make haystack slow : for a 13 second pipeline, it takes extra 6 seconds when running with langfuse tracer for a total of 19s.
After profiling my pipeline, it looks like uploading to langfuse take time, as if it was waiting for it to be uploaded to run the next component.
I'm using langfuse self hosted
Describe your environment (please complete the following information):
- OS: ubuntu (docker)
- Haystack version: 2.9
- Integration version: 0.7
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Langfuse tracer integration and reproduce the reported 13-second pipeline taking about 19 seconds with tracing enabled. Profile the pipeline to confirm whether Langfuse uploads block the next component, then compare the traced and untraced timings; done means the source of the extra delay is identified and the integration no longer adds the reported blocking overhead.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100