refly-ai / refly-ai/refly

Invalid image input when using Ollama as model backend

Open
#647 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
7.5k
Forks
729
PR merge metrics
No merged PRs in 30d

Description

Describe the bug
docker本地部署接入Ollama视觉模型gemma3:12b分析图片报错,报错信息如下:
Error: 400 invalid image input
at APIError.generate (/app/node_modules/.pnpm/openai@4.83.0_ws@8.17.1_zod@3.23.8/node_modules/openai/error.js:45:20)
at OpenAI.makeStatusError (/app/node_modules/.pnpm/openai@4.83.0_ws@8.17.1_zod@3.23.8/node_modules/openai/core.js:299:33)
at OpenAI.makeRequest (/app/node_modules/.pnpm/openai@4.83.0_ws@8.17.1_zod@3.23.8/node_modules/openai/core.js:343:30)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /app/node_modules/.pnpm/@langchain+openai@0.4.3_@langchain+core@0.3.29_ws@8.17.1/node_modules/@langchain/openai/dist/chat_models.cjs:1519:29
at async RetryOperation._fn (/app/node_modules/.pnpm/p-retry@4.6.2/node_modules/p-retry/index.js:50:12)

To Reproduce
Steps to reproduce the behavior:

  1. 上传图片节点(图片格式 png,大小800kb左右)
  2. 使用AI问答节点并将图片节点作为上下文

Screenshots
Image

Desktop (please complete the following information):

  • OS: Mac OS M1 Pro
  • Browser edge
  • Version 15.3.2

Additional context
docker desktop: 4.32.2

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Ollama model-backend integration and the image-upload-to-AI-Q&A context path described in the reproduction steps. Reproduce the request with a PNG through Docker, then trace the payload into the OpenAI-compatible client; done means the Ollama vision model accepts the image without an "invalid image input" error.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, ollama, typescript
Domain
ai, api, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.