ianarawjo / ianarawjo/ChainForge
Feature request: Enable image inputs for Custom Providers (vision parity + simple toggle)
- Dominant language
- TypeScript
- Stars
- 3k
- Forks
- 256
- Avg merge
- 4h 40m
- Merged PRs (30d)
- 22
Description
Hi ChainForge team — huge thanks for the amazing tool and the new Media Node image-input workflow. It’s been great with the built-in providers. 🙏
**What I’m trying to do**
Use a **Custom Provider** that wraps a third-party model which **does support vision** in other popular clients. With built-in providers (e.g., GPT-4o / Gemini / Claude / Ollama), image understanding works as expected in ChainForge.
**Repro steps (minimal, product-level)**
1. Add a **Media Node**, upload a real-life photo (e.g., a cat).
2. In a **Prompt Node**, include `{image}` so I can connect the Media Node → Prompt Node.
3. Run with a **built-in vision model** → ✅ It correctly says it’s a cat (or otherwise describes the photo).
4. Switch to my **Custom Provider** (wrapping a vision-capable third-party model) and ask the same question like “What’s in this image?” → ❌ It returns unrelated text as if no image was sent (e.g., says “this is a Python tutorial”), i.e., **hallucinations** consistent with the image not being received.
**Expected**
Custom Providers receive the **same image input** that built-in providers do, so vision-capable third-party models can recognize what’s in the photo.
**Actual**
When using a Custom Provider, responses look like the model **didn’t get the image at all**, leading to off-topic/hallucinated text.
**Why I think this matters / related notes**
Release notes for v0.3.6 say image inputs are currently limited to **OpenAI, Anthropic, Google, and Ollama**. This matches what I’m seeing (works with built-ins, not with Custom Provider). I’m hoping Custom Providers can get parity here without me having to modify code. ([[GitHub](https://github.com/ianarawjo/ChainForge/releases)][1])
**Requests (feature & docs)**
* Please **enable image inputs for Custom Providers** so they can receive Media Node images just like built-ins.
* If possible, make this a simple **UI/setting toggle**, e.g., “**Image support: enable/disable**” for a Custom Provider—so users don’t need to patch code.
* If this is already possible, could you add **user-facing documentation** that explains the **recommended way** to get Media Node images into a Custom Provider (no deep implementation details needed—just how to turn it on or what option/parameter to use)?
**Environment**
* ChainForge version: **v0.3.6**
* Install: local (`pip install chainforge` → `chainforge serve`)
* OS/Browser: Windows 11 + Chrome (also reproducible on my setup)
* Custom Provider: wraps a third-party model that handles vision correctly in other clients
Thanks again for all your work on ChainForge — the Media Node feature is already super helpful, and adding **Custom Provider image support** (ideally via a simple toggle) would make it even more powerful. ❤️
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.