anomalyco / anomalyco/opencode
Feature: Dynamic context length detection for OpenAI-compatible providers
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 209k
- Forks
- 27.5k
- PR merge metrics
- PR metrics pending
Description
Summary
When using a custom OpenAI-compatible provider (via @ai-sdk/openai-compatible), opencode does not dynamically detect the context length of the currently loaded model from the server's /models endpoint. Instead, it requires limit.context to be hardcoded in the config for each model.
Problem
Local model servers (LM Studio, Unsloth Studio, etc.) expose context_length for the loaded model in their /models API response. However, opencode ignores this and requires static configuration. This means:
- Users must manually set
limit.contextfor every model - If the user changes the context size on the server, opencode doesn't know
- Different models may have different context lengths, requiring constant config updates
Expected Behavior
opencode should query the provider's /models endpoint at runtime and read the context_length field to determine the appropriate context window for the currently loaded model.
Example Server Response
{
"object": "list",
"data": [
{
"id": "DavidAU/Qwen3.5-9B-...",
"object": "model",
"owned_by": "unsloth-studio",
"context_length": 262144,
"max_context_length": 208128,
"native_context_length": 262144,
"loaded": true
}
]
}
Current Workaround
Users must manually specify limit.context (and limit.output) in opencode.json for each model definition under provider.<name>.models.
Suggested Solution
For OpenAI-compatible providers, fetch the /models endpoint at connection time and use the context_length from the loaded model's metadata as the context window. This could be refreshed periodically or on model switch.
Environment
- Provider type:
@ai-sdk/openai-compatible - Local server: Unsloth Studio (also applies to LM Studio, Ollama, etc.)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the @ai-sdk/openai-compatible provider integration and how provider..models configuration supplies limit.context. Then inspect how the OpenAI-compatible /models response is accessed or could be queried at connection time. Done means the loaded model's context_length is used dynamically, including after a model switch or server-side context-size change, without requiring a hardcoded value.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- api, backend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100