Chat + image attachment fails: "URL scheme must be http, https, or data, got aipg-media:"
- Dominant language
- TypeScript
- Stars
- 979
- Forks
- 132
- Avg merge
- 9h 23m
- Merged PRs (30d)
- 8
Description
**Description:**
```markdown
In AI Playground **3.2.0-beta**, sending a **text + image** message in Chat mode fails immediately with:
```
URL scheme must be http, https, or data, got aipg-media:
```
followed by:
```
AI_NoOutputGeneratedError: No output generated. Check the stream for errors.
```
Text-only chat works fine. The failure only happens when an image is attached.
Root cause (from inspecting `app.asar`):
1. When an image is attached in Chat, the UI stores a file part using the custom media protocol:
- `{ type: "file", mediaType: "image/png", url: "aipg-media://media/...", filename: "..." }`
2. Before calling the model, there is conversion logic intended to turn `aipg-media://` URLs into `data:` URIs via `readAipgMediaAsBase64`. That logic still checks the **old** AI SDK shape:
```javascript
e.type === "file"
&& e.mediaType?.startsWith("image/")
&& typeof e.data === "string" // <-- false with current AI SDK
&& e.data.startsWith("aipg-media://")
```
3. The bundled AI SDK now normalizes file data into a **structured object**:
```javascript
// convertToModelMessages → mw(e.data)
{ type: "file", data: { type: "url", url: "aipg-media://media/..." }, mediaType: "image/png" }
```
Because `typeof e.data === "object"` (not `"string"`), the aipg-media → data-URI conversion is skipped.
4. AI SDK then validates the download URL (`validateDownloadUrl`) and rejects the custom scheme:
```text
URL scheme must be http, https, or data, got aipg-media:
```
So this looks like a format-mismatch regression between the chat image pipeline and the bundled AI SDK message model, not a local install/config problem.
## To Reproduce
Steps to reproduce the behavior:
1. Open AI Playground 3.2.0-beta on Windows.
2. Select Chat mode with a **vision-capable** local model (llama.cpp / OpenVINO backend).
3. Attach any local image (PNG/JPG) via the prompt-area `+` button or drag-and-drop.
4. Type any short prompt (e.g. "Describe this image").
5. Click Send.
6. See error toast/console:
```
[inference] inference/stream-failed: URL scheme must be http, https, or data, got aipg-media:
```
and
```
AI_NoOutputGeneratedError: No output generated. Check the stream for errors.
```
7. Remove the image and send text-only with the same model → succeeds.
## Expected behavior
Chat with an attached image should:
1. Resolve `aipg-media://` to a base64 `data:` URI (or equivalent inline bytes) **before** handing the message to the AI SDK / local backend.
2. Send the multimodal request successfully, same as text-only chat.
The existing conversion helper already knows how to do step 1 (`gE()` / `readAipgMediaAsBase64`); it just never runs because it still matches the old `data: string` shape.
### Suggested fix direction
Update the pre-request conversion (and the “has aipg-media image” detector) to also handle the current AI SDK shape, for example:
```javascript
// Current (misses structured data):
typeof e.data === "string" && e.data.startsWith("aipg-media://")
// Also match:
e.data?.type === "url" && typeof e.data.url === "string" && e.data.url.startsWith("aipg-media://")
// and/or
typeof e.url === "string" && e.url.startsWith("aipg-media://")
```
Then replace it with a data URI / `{ type: "data", data: base64 }` payload before `streamText` / chat transport.
## Screenshots
N/A (console error only). Full stack from renderer:
```
URL scheme must be http, https, or data, got aipg-media:
Error: URL scheme must be http, https, or data, got aipg-media:
at dist-BRzcC_7v.js:116:14657
at dist-BRzcC_7v.js:117:10246
at Array. (dist-BRzcC_7v.js:116:59168)
at kD.processQueue (dist-BRzcC_7v.js:116:59048)
...
at LD.sendMessage (dist-BRzcC_7v.js:117)
...
[unknown] global/unhandled-rejection: No output generated. Check the stream for errors.
AI_NoOutputGeneratedError: No output generated. Check the stream for errors.
```
## Environment (please complete the following information):
- OS: Windows 11
- GPU: Intel(R) Arc(TM) B580 Graphics (device id 0xE20B)
- CPU: Intel(R) Core(TM) i9-14900K (3.20 GHz)
- Version: AI Playground 3.2.0-beta
- Backend: llama.cpp chat backend (also seen with OpenVINO backend configured)
- Model: Qwen3.5-9B-Q4_K_M.gguf
- Install path: `E:\AI Playground`
- Product mode: studio
- Reproducibility: 100% when an image is attached; 0% for text-only
## Additional context
- Text-only chat with the same model/backend is stable.
- Unrelated noise in the same log session (not the cause of this bug):
- ComfyUI `insightface` wheel download from GitHub timed out (`uv.check-details.ComfyUI`).
- ComfyUI venv “environment doesn't match expected state” warning.
- The custom protocol is registered as privileged in the Electron main process (`registerSchemesAsPrivileged([{ scheme: "aipg-media", ... }])`), so the UI can preview images locally; the chat request path just never rewrites that URL for the model.
- This blocks any multimodal chat use case on 3.2.0-beta and makes image understanding unusable until fixed.
Contributor guide
Research direction
Start by locating the Chat image-message conversion and aipg-media detector in the source corresponding to the bundled app.asar; inspect readAipgMediaAsBase64 and the data shape passed before streamText or the chat transport. Reproduce with a local image and a vision-capable model, then verify that the structured URL is converted before AI SDK validation and that text-only chat remains working.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- ai, api
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100