Abraxas-365 / Abraxas-365/langchain-rust
Chain does not support images
- 主要語言
- Rust
- 星號
- 1.3k
- 分支
- 177
- PR 合併指標
- 30 天內沒有已合併 PR
描述
**Describe the bug**
Using chain (`ConversationalChainBuilder` or `Agent`) does not support images.
```rust
prompt_args! {
"input" => input,
}
```
does not allow to pass an `image url`.
If the `url` is passed as a string the in input, an error is thrown `OpenAI error: stream failed: Invalid status code: 400 Bad Request` probably because the image is not under the right structure ([`Image content part`](https://platform.openai.com/docs/guides/vision)).
Also
`langchain_rust::schemas::Message::messages_to_string` only serialize text content. It does not take in consideration images.
https://github.com/Abraxas-365/langchain-rust/blob/891c56f0e2f2a98eed883f906c3de6cb82770797/src/schemas/messages.rs#L149-L155
**To Reproduce**
```rust
let chain = ConversationalChainBuilder::new()
.llm(llm)
.build();
let base64_image_url = "data:image/jpeg;base64,23434234234234".to_string();
let input = format!("What is that? {}", base64_image_url);
let result = chain.invoke(prompt_args! {
"input" => input,
});
```
**Expected behavior**
A way to pass image to a chain or at minimum an Agent.
Since images are on `langchain_rust::schemas::Message` structure, I would expect them to work in every aspect of the library.
**Additional context**
This is some kind of a violation of the `Liskov Principle` & `Interface Segregation Principles`. If image are available on `langchain_rust::schemas::Message` they should be supported everywhere in the library.
貢獻指南
研究方向
The issue points to `langchain_rust::schemas::Message::messages_to_string` in src/schemas/messages.rs, which currently only serializes text content. Start by examining the Message struct to understand how image content is stored. Then, look at ConversationalChainBuilder and Agent to see how messages are processed and where image support needs to be added. Testing will involve creating a chain with an image URL and ensuring it conforms to OpenAI's vision API structure.
由索引模型根據 Issue 內容生成。
評估
- 技術堆疊
- rust
- 領域
- ai-infra-agents
- Issue 類型
- 缺陷
- 難度
- 4/5
- 預估耗時
- 3-5 天
- 活躍度
- 停滯
- 描述清晰度
- 描述清楚
- 新手友好度
- 45/100