Feature: configurable effective context-window ceiling (below model max) that auto-compact targets
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 19.9k
- Forks
- 2.3k
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 30
Description
Summary
Feature request: a configurable effective context-window ceiling that can be set lower than a model's native max context, with auto-compact triggering relative to that ceiling — not the model's raw context_window().
Current behavior
Provider::context_window()resolves to the model's native max (e.g. 1,000,000 forclaude-opus-5, 200,000 forclaude-sonnet-4-5) — seecrates/jcode-provider-anthropic-runtime/src/context_window.rs.jcode-compaction-core::COMPACTION_THRESHOLD(0.80) andCRITICAL_THRESHOLD(0.95) are hardcoded fractions applied directly to that native max (crates/jcode-base/src/compaction.rs).- The only existing user override,
context_window(aliascontext_limit) in[[providers.<name>.models]], is documented as filling in a value when the provider has no usable/v1/modelscatalog response — it's meant to report the true max, not to cap it lower for operational reasons.
There's currently no way to say "this model supports 1M tokens, but I want jcode to always compact once we hit e.g. 100K, regardless."
Why this matters
On models with very large native windows (e.g. Opus 5 at 1M), a session can run for a very long time and accumulate a huge amount of context before the 80%/95% thresholds ever fire. In practice this means:
- Much higher per-turn cost and latency as the transcript grows, well before compaction ever kicks in.
- Degraded model attention/quality on long, sprawling contexts even when technically "within budget."
- Less frequent, much larger compaction events instead of smaller, more frequent ones — worse for keeping a tight working set.
Being able to set a lower operational ceiling (e.g. "treat this model as if it only has 120K tokens of context, even though it can do 1M") lets a user trade off cost/quality/latency deliberately, independent of what the model can technically hold.
Suggested fix
- Add a config option (e.g.
effective_context_windoworcontext_window_cap, global and/or per-provider/per-model) that, when set, is used as the base forCOMPACTION_THRESHOLD/CRITICAL_THRESHOLDmath instead ofprovider.context_window(). - Should be clearly distinguished from the existing
context_window/context_limitfield (which reports the model's true capacity for catalog-less providers) — this is a policy cap, not a capability report. - The TUI context-usage meter should reflect usage against the capped ceiling when set, so the displayed percentage matches when auto-compact will actually fire.
- Reasonable to validate that the cap is
<=the model's actual resolved context window (silently clamp or warn, not silently ignored).
Related code pointers
crates/jcode-compaction-core/src/lib.rs:9,13—COMPACTION_THRESHOLD/CRITICAL_THRESHOLDconstants.crates/jcode-base/src/compaction.rs— threshold application (try_auto_compact_after_context_limit, hard/soft threshold wait logic).crates/jcode-provider-anthropic-runtime/src/context_window.rsand equivalents in other provider-runtime crates —context_window()resolution.crates/jcode-config-types/src/lib.rs:353—CompactionConfigstruct, natural home for a new cap field.crates/jcode-config-types/src/lib.rs:422—NamedProviderModelConfig— existingcontext_window/context_limitfield for comparison/contrast.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with CompactionConfig in crates/jcode-config-types/src/lib.rs:353 and compare NamedProviderModelConfig at line 422, then trace threshold application in crates/jcode-base/src/compaction.rs. Review context_window() resolution in the provider-runtime crates and the TUI context-usage meter. Done means a validated operational ceiling can be configured, auto-compaction uses it, and displayed usage reflects the same ceiling.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- ai, cli, tooling
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100