Support configuring context window size (`num_ctx`) for local models in `ChatOllama`
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 176
- Forks
- 28
- Avg merge
- 18h 42m
- Merged PRs (30d)
- 16
Description
When using ChatOllama in chatlas to talk to local models (especially models with reasoning capabilities like gemma4 or larger parameter models), responses are easily truncated due to Ollama's default context window limit of 2048 tokens.
Because chatlas (seemingly) interfaces with Ollama via the OpenAI-compatible /v1/chat/completions API and strictly filters parameters in translate_model_params(), there is currently no way for users to increase the context window size (num_ctx) on-the-fly.
In contrast, the R equivalent (ellmer::chat_ollama) handles this seamlessly by communicating directly with Ollama's native /api/chat API, which natively supports setting num_ctx and num_predict via the request options.
Reproducible Examples (Reprex)
1. R (ellmer) - Success
Using ellmer, we can configure both max output tokens and a large context window, avoiding truncation:
library(ellmer)
# ellmer maps these parameters directly to Ollama's options block
chat <- chat_ollama(
model = "gemma4",
params = params(
max_tokens = 2048,
num_ctx = 8192
)
)
response <- chat$chat("Write a very long essay about the history of statistics.")
cat(nchar(response)) # Successfully generates a long, complete response
2. Python (chatlas) - Truncation (Failure)
Using chatlas, setting max_tokens is supported, but we cannot pass num_ctx to the client. When the prompt + response (including thinking traces) exceeds 2048 tokens, it gets truncated:
from chatlas import ChatOllama
# Initialize client (no way to configure context window)
chat = ChatOllama(model="gemma4")
chat.set_model_params(max_tokens=2048)
# Writing a long prompt/response will hit Ollama's default 2048 context limit
response = chat.chat("Write a very long essay about the history of statistics.")
print(chat.get_turns()[-1].finish_reason) # prints "length" (truncated early)
Proposed Solutions
-
Allow custom parameters in
ChatOllama:
Exposenum_ctx(or a genericoptionspayload dictionary) in theChatOllamaclient initialization, and have the underlyingOllamaProviderpass it along as part of the request payload to Ollama's completion endpoint (Ollama's/v1/chat/completionsaccepts custom parameters likenum_ctxin the JSON request body). -
Expose provider-specific parameters in
set_model_params():
Instead of raising aTypeErrorfor non-standard parameters, allow provider-specific keys likenum_ctxto be passed through to the underlying request payload.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing ChatOllama through translate_model_params() into OllamaProvider and review how set_model_params() filters arguments for the /v1/chat/completions payload. Compare the proposed num_ctx or options handling with Ollama's supported request fields. Done means users can configure num_ctx for local models without a TypeError and the value reaches the request payload.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- ollama, python
- Domain
- ai, api
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100