huggingface / huggingface/AnyLanguageModel

Make `LanguageModelSession` transcript observable while streaming responses

Open
#103 8 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Swift
Stars
935
Forks
94
Avg merge
2d 6h
Merged PRs (30d)
55

Description

As described by @belozierov in https://github.com/mattt/AnyLanguageModel/issues/98#issuecomment-3830901075:

> During an LLM request, the model may perform many turns, producing intermediate outputs such as reasoning, user-visible messages, tool calls, etc. Ideally, these intermediate steps should be observable by the user via `LanguageModelSession`.
>
> For this to work, it would be useful to update `LanguageModelSession.transcript` while `LanguageModel.streamResponse` is running.
>
> One possible approach would be for `LanguageModel.streamResponse` to return not just a stream of `Content`, but an enum that can represent either `Content` or `Transcript.Entry`, where `Transcript.Entry` values would be appended to `LanguageModelSession.transcript` during the stream.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.