FormulaMonks / FormulaMonks/kurt

Add high-level looping function based on `generateWithOptionalTools`.

Open
#25 0 comments 0 reactions 1 assignee Claimed by @jemc View on GitHub
enhancement
Dominant language
TypeScript
Stars
4
Forks
0
PR merge metrics
No merged PRs in 30d

Description

One common pattern for LLM apps is to keep calling `generateWithOptionalTools` until the LLM has decided it's used enough tools to finally generate a natural language output.

This pattern is subject to a halting problem (the LLM may never decide it is satisfied, and may not be making progress toward a discernable goal - it may even lose track of the goal over time).

However the pattern is common enough (especially in quick prototypes) that it's worth supporting a simple way of doing it. Our [current code sample for this](https://github.com/FormulaMonks/kurt/tree/6846567795f87388f3e1959dac08eb6386c08072/packages/kurt#generate-with-optional-tools) is kinda ugly and verbose, so we'd like to provide a high-level looping function in `Kurt` that can accept:

- the standard options for generating (prompt, extraMessages, etc)
- a map of named tool parameter schemas
- a map of named tool functions that take those data types as inputs
- some additional settings, including:
- a maximum number of iterations to allow before calling a final time with no tools (to force natural language output)
- perhaps a way to remove tools that have already been called? (or been called a certain number of times?)
- perhaps some control over if/how/what messages get added into the extraMessages history
- whatever other options make sense

The output should get streamed just like we do now with `KurtStream`. However we need some way to have multiple consecutive streams, because we'll get multiple finished messages for each stream. Maybe an `AsyncIterable` is the right return type.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.