Feature request: let users register vision-capable models in provider config
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 19.9k
- Forks
- 2.3k
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 30
Description
Problem
jcode decides image support per provider, not per model. On the OpenAI-compatible runtime, supports_image_input() returns true for every non-ChatGPT-web model. The only place to say a specific model handles images is the models[].input list in a named provider profile, and only the OpenRouter runtime reads it. So a text-only model on an OpenAI-compatible endpoint gets image blocks sent to it and the request fails, and a vision model that jcode has no catalog entry for can never be marked as vision-capable.
What I want
A way to register a model as vision-capable in config, and have that flag drive supports_image_input() everywhere.
- Honor the existing
models[].inputfield for OpenAI-compatible named providers, not just OpenRouter. - A model with
input: ["text", "image"]reports vision support. A model withinput: ["text"], or one that is not listed, falls back to the provider default. - A provider-level default would help too, something like
image_input: true | false | auto, so users do not have to list every model just to flip the default.
Why it matters
Issue #755 shows the failure mode: a text-only model receives an image block and the session breaks. The opposite problem exists too. A vision model behind a gateway with no jcode catalog entry cannot be advertised as vision-capable, so screenshots and pasted images are stripped or rejected even though the model could handle them.
Notes
Provider::supports_image_input()already exists on the trait.NamedProviderModelConfig.inputalready exists incrates/jcode-config-types/src/lib.rs.- The OpenRouter runtime maps
models[].inputto image support, but the plain OpenAI runtime never consults it.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with Provider::supports_image_input() and NamedProviderModelConfig in crates/jcode-config-types/src/lib.rs, then compare how the OpenRouter runtime maps models[].input with the plain OpenAI runtime. Done means named OpenAI-compatible models honor text/image input declarations, unlisted models retain provider defaults, and provider-level defaults work consistently across runtimes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- ai, api
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100