posit-dev / posit-dev/chatlas

Add on_request_start() / on_request_end() hooks to Chat

Open
#413 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

ai-triage:done Priority: Low
Dominant language
Python
Stars
176
Forks
28
Avg merge
18h 42m
Merged PRs (30d)
16

Description

Summary

ellmer recently added on_request_start() / on_request_end() hooks on Chat (tidyverse/ellmer#1052). These fire around each model request within the tool-calling loop (not just around individual tool calls), letting callers inspect or rewrite the conversation between tool rounds within a single chat()/chat_async() call — e.g. to compact turns as the context window fills up.

chatlas currently only exposes tool-level callbacks: Chat.on_tool_request() and Chat.on_tool_result() (see chatlas/_chat.py). There's no equivalent hook that fires before/after each model request in the tool loop.

Proposed behavior (mirroring ellmer)

  • on_request_start(callback): callback receives the list of turns about to be sent, fired at the top of each tool-loop iteration (before the request is dispatched). Return value (if any) could allow rewriting the turns, mirroring ellmer's design.
  • on_request_end(callback): callback receives the resulting turn after each model request completes.
  • Should fire correctly in both sync (chat()) and async (chat_async()) paths, and work whether or not a tool loop actually runs.

References

  • ellmer PR: tidyverse/ellmer#1052
  • chatlas tool-level callbacks for comparison: chatlas/_chat.py (on_tool_request, on_tool_result, _on_tool_request_callbacks, _on_tool_result_callbacks)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in chatlas/_chat.py by reading on_tool_request(), on_tool_result(), their callback storage, and the chat() and chat_async() paths. Compare the proposed lifecycle with tidyverse/ellmer#1052, then define the start and end callback behavior for each model request, including tool-loop and no-tool cases. Done means both paths expose and invoke the hooks consistently.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend-api-design
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.