Abraxas-365 / Abraxas-365/langchain-rust

Add support for cancellation

未关闭
#75 2 条评论 3 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
Rust
星标
1.3k
派生
177
PR 合并指标
30 天内没有已合并 PR

描述

Using LLMs can be expensive or slow. Add support for cancellation - https://docs.rs/tokio-util/latest/tokio_util/sync/struct.CancellationToken.html

Seems like in JS, they use the 2nd args for options - https://js.langchain.com/docs/modules/model_io/chat/cancelling_requests

could also support none as option.

```rust
let open_ai = OpenAI::default();
let response = open_ai.invoke("What is the capital of France?", None).await.unwrap();
```

```rust
let token = CancellationToken::new();

let open_ai = OpenAI::default();
let response = open_ai.invoke("What is the capital of France?", InvokeOptions::default().with_cancellation(token.clone())).await.unwrap();
```

贡献指南

打开贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。