Abraxas-365 / Abraxas-365/langchain-rust
OpenAI stream error
- Lenguaje dominante
- Rust
- Estrellas
- 1.3k
- Forks
- 177
- Métricas de merge de PR
- Sin PR fusionados en 30 d
Descripción
**Describe the bug**
When using some OpenAI-compatible LLM providers, the content of the stream is sometimes empty when using the OpenAI client
```
StreamData {
value: Object {
"choices": Array [
Object {
"delta": Object {
content": String(":“"),
"function_call": Null,
"refusal": Null,
"role": String("assistant"),
"tool_calls": Null,
},
"finish_reason": Null,
"index": Number(0),
"logprobs": Null,
},
],
"created": Number(1740119495),
"id": String("0195273454f7cf3b20d629a8d0a456d5"),
"model": String("deepseek-ai/DeepSeek-R1-Distill-Qwen-7B"),
"object": String("chat.completion.chunk"),
"service_tier": Null,
"system_fingerprint": String(""),
"usage": Object {
"completion_tokens": Number(118),
"completion_tokens_details": Null,
"prompt_tokens": Number(324),
"prompt_tokens_details": Null,
"total_tokens": Number(442),
},
},
tokens: Some(
TokenUsage {
prompt_tokens: 324,
completion_tokens: 118,
total_tokens: 442,
},
),
content: "",
}
```
**To Reproduce**
```
let mut stream = match chain.stream(prompt_args! {
"chat_history" => chat_history,
"input" => message.content.clone(),
"documents" => documents
}).await {
Ok(stream) => stream,
Err(e) => {
return;
}
};
while let Some(result) = stream.next().await {
match result {
Ok(value) => {
dbg!(value.clone());
},
Err(e) => {
panic!("Error invoking LLMChain: {:?}", e)
},
}
}
```
**Expected behavior**
the content has value
**Additional context**
the root cause is that when the stream method implemented by Open AI Client ignores the content after parsing the usage of token.
https://github.com/Abraxas-365/langchain-rust/blob/60c71258d8a40c278f2867f6b4d3871265f6e638/src/llm/openai/mod.rs#L168-L203
Guía de contribución
Evaluación
Este issue todavía no se ha evaluado.