spring-projects / spring-projects/spring-ai

Feature request: provider-agnostic Model Capabilities registry (context window, token limits, feature support)

Open
#6,486 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

status: waiting-for-triage
Dominant language
Java
Stars
9.5k
Forks
2.9k
Avg merge
1d 7h
Merged PRs (30d)
6

Description

Expected Behavior

Provide a provider-agnostic, runtime-accessible model capabilities registry so applications can discover per-model capability and limits (for example: context window, max output tokens, and support for tool calling, streaming, multimodality, reasoning). The API should be additive and opt-in for providers that can supply this data.

Example API sketch (consumer):

Optional<ModelCapabilities> caps = modelCapabilitiesProvider.getCapabilities("gpt-5-mini");
if (caps.isPresent() && caps.get().contextWindowTokens() != null) {
    validatePromptBudget(caps.get().contextWindowTokens());
}
if (caps.isPresent() && !caps.get().supportsToolCalling()) {
    // Disable tool execution for this model at runtime
}

Providers or model implementations may register capabilities beans or the framework may ship a central lookup that aggregates provider-supplied metadata.

Current Behavior

Spring AI exposes model operations and options but does not provide a unified runtime capability descriptor for models. As a result, application teams must:

  • hardcode per-model limits in their apps, or
  • maintain and synchronize an external map/registry of model -> capabilities, or
  • consult provider documentation at development time and rely on runtime inference.

None of these approaches are ideal for production systems that must validate prompt sizes, set safe defaults, or degrade features dynamically.

Context

How has this affected you?

  • It increases application complexity and risk: apps either risk runtime failures (oversized prompts) or conservative limits that reduce utility.
  • It forces duplicate maintenance across projects and teams.

What are you trying to accomplish?

  • Add a small, provider-neutral contract to Spring AI so client code can query model capabilities at runtime and adapt behavior safely.

What other alternatives have you considered?

  • Keep capability registries in application code (current workaround) — error-prone and hard to keep current.
  • Provider-specific metadata APIs only — workable but fragments application logic and increases branching.
  • Documentation-only guidance — insufficient for runtime validation.

Additional notes

  • Fields should be nullable/optional when a provider cannot return them.
  • Providers may expose extra, provider-specific keys via a map for advanced use cases.
  • I can follow up with a PR that implements the minimal contract and a provider-side stub for one or two model integrations to demonstrate the pattern.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing Spring AI's existing model operations and options, then compare how provider integrations expose model metadata. Define the minimal optional, provider-neutral capabilities contract and runtime lookup, including provider-specific extensions if supported. Done means client code can query capabilities safely and a provider-side stub demonstrates the pattern.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spring
Domain
ai, backend-api-design
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.