deepset-ai / deepset-ai/haystack

Updates to the ChatGenerator protocol

Open
#10,992 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

P2
Dominant language
Python
Stars
26.6k
Forks
3.2k
Avg merge
1d 3h
Merged PRs (30d)
194

Description

Two ideas for updating the ChatGenerator protocol:

  1. Update the return type of the run method to at least require replies: list[ChatMessage] since we rely on that key being present when we use the protocol in the codebase. This should be possible through using TypedDict, total=False and Required (still needs to be tested). So something like
from typing import Protocol
from typing_extensions import Required, TypedDict
from haystack.dataclasses import ChatMessage


class ChatGeneratorOutput(TypedDict, total=False):
    """
    Return type for Chat Generators.

    ``replies`` is always required. Implementations may include additional keys.
    """
    replies: Required[list[ChatMessage]]


class ChatGenerator(Protocol):
    def run(self, messages: list[ChatMessage]) -> ChatGeneratorOutput:
  1. Consider adding a run_async to the protocol since we have finished adding run_async to all of our chat generators in core and in core-integrations https://github.com/deepset-ai/haystack/issues/8858 and https://github.com/deepset-ai/haystack-core-integrations/issues/1379. Only a consideration since it's not guaranteed that community made components follow the protocol, but we are starting to add run_async methods to components that use the ChatGenerator protocol so I wonder if creating a second protocol (e.g. ChatGeneratorWithAsync) would be an alternative.

👋 Hello there! This issue will be handled internally and isn't open for external contributions. If you'd like to contribute, please take a look at issues labeled contributions welcome or good first issue. We'd really appreciate it!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the ChatGenerator protocol and its implementations, then review how run return values and run_async methods are used across the codebase. Check issues #8858 and #1379 for the existing async work. Done would mean an agreed protocol design with its typing and async implications addressed, though the issue states that this work is handled internally.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend-api-design
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.