Abraxas-365 / Abraxas-365/langchain-rust
Add support for cancellation
未关闭
- 主要语言
- Rust
- 星标
- 1.3k
- 派生
- 177
- PR 合并指标
- 30 天内没有已合并 PR
描述
Using LLMs can be expensive or slow. Add support for cancellation - https://docs.rs/tokio-util/latest/tokio_util/sync/struct.CancellationToken.html
Seems like in JS, they use the 2nd args for options - https://js.langchain.com/docs/modules/model_io/chat/cancelling_requests
could also support none as option.
```rust
let open_ai = OpenAI::default();
let response = open_ai.invoke("What is the capital of France?", None).await.unwrap();
```
```rust
let token = CancellationToken::new();
let open_ai = OpenAI::default();
let response = open_ai.invoke("What is the capital of France?", InvokeOptions::default().with_cancellation(token.clone())).await.unwrap();
```
贡献指南
评估
这个 Issue 还没有评估数据。