1jehuang / 1jehuang/jcode

Feature request: let users register vision-capable models in provider config

Open
#772 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

autonomous: no enhancement priority: medium triage: needs-decision
Dominant language
Rust
Stars
19.9k
Forks
2.3k
Avg merge
2d 7h
Merged PRs (30d)
30

Description

Problem

jcode decides image support per provider, not per model. On the OpenAI-compatible runtime, supports_image_input() returns true for every non-ChatGPT-web model. The only place to say a specific model handles images is the models[].input list in a named provider profile, and only the OpenRouter runtime reads it. So a text-only model on an OpenAI-compatible endpoint gets image blocks sent to it and the request fails, and a vision model that jcode has no catalog entry for can never be marked as vision-capable.

What I want

A way to register a model as vision-capable in config, and have that flag drive supports_image_input() everywhere.

  • Honor the existing models[].input field for OpenAI-compatible named providers, not just OpenRouter.
  • A model with input: ["text", "image"] reports vision support. A model with input: ["text"], or one that is not listed, falls back to the provider default.
  • A provider-level default would help too, something like image_input: true | false | auto, so users do not have to list every model just to flip the default.

Why it matters

Issue #755 shows the failure mode: a text-only model receives an image block and the session breaks. The opposite problem exists too. A vision model behind a gateway with no jcode catalog entry cannot be advertised as vision-capable, so screenshots and pasted images are stripped or rejected even though the model could handle them.

Notes

  • Provider::supports_image_input() already exists on the trait.
  • NamedProviderModelConfig.input already exists in crates/jcode-config-types/src/lib.rs.
  • The OpenRouter runtime maps models[].input to image support, but the plain OpenAI runtime never consults it.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with Provider::supports_image_input() and NamedProviderModelConfig in crates/jcode-config-types/src/lib.rs, then compare how the OpenRouter runtime maps models[].input with the plain OpenAI runtime. Done means named OpenAI-compatible models honor text/image input declarations, unlisted models retain provider defaults, and provider-level defaults work consistently across runtimes.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
ai, api
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.