Abraxas-365 / Abraxas-365/langchain-rust
Add support for cancellation
- Lingua principale
- Rust
- Stelle
- 1.3k
- Fork
- 177
- Metriche di merge delle PR
- Nessuna PR unita negli ultimi 30g
Descrizione
Using LLMs can be expensive or slow. Add support for cancellation - https://docs.rs/tokio-util/latest/tokio_util/sync/struct.CancellationToken.html
Seems like in JS, they use the 2nd args for options - https://js.langchain.com/docs/modules/model_io/chat/cancelling_requests
could also support none as option.
```rust
let open_ai = OpenAI::default();
let response = open_ai.invoke("What is the capital of France?", None).await.unwrap();
```
```rust
let token = CancellationToken::new();
let open_ai = OpenAI::default();
let response = open_ai.invoke("What is the capital of France?", InvokeOptions::default().with_cancellation(token.clone())).await.unwrap();
```
Guida per i contributori
Apri la guida per i contributori
Valutazione
Questa issue non è ancora stata valutata.