spring-projects / spring-projects/spring-ai

Normalize / categorize LLM finish reasons across providers in ChatGenerationMetadata (keep raw reason)

Open
#5,146 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement model client
Dominant language
Java
Stars
9.5k
Forks
2.9k
Avg merge
1d 7h
Merged PRs (30d)
6

Description

Actual behavior

ChatGenerationMetadata.getFinishReason() exposes the provider-specific finish reason as a raw String.
In multi-provider setups the values differ (e.g. stop, end_turn, STOP, length, max_tokens, MAX_TOKENS, content_filter, SAFETY, etc.), so applications must implement their own mapping logic to build provider-agnostic audits/metrics/alerts. This leads to duplicated, brittle code and inconsistent semantics across projects.

Expected behavior

Spring AI should provide a normalized / categorized finish reason in addition to the raw provider value.

Example (additive, backward compatible):

enum FinishReasonCategory {
  COMPLETED,   // normal stop/end_turn/STOP
  TRUNCATED,   // length/max_tokens/MAX_TOKENS/context_window_exceeded
  TOOL_CALL,   // tool_calls/tool_use
  FILTERED,    // content_filter/SAFETY/refusal/SPII/PROHIBITED_CONTENT/RECITATION
  OTHER, UNKNOWN
}

interface ChatGenerationMetadata {
  String getFinishReason();                 // existing raw reason (unchanged)
  FinishReasonCategory getFinishCategory(); // new normalized category
}

Provider modules would map their native reasons to the category while preserving raw. Unknown/new values should map to UNKNOWN but keep the raw string intact.

This would make it much easier to:

  • colorize/flag audit logs consistently (green=COMPLETED, red=TRUNCATED/FILTERED, yellow=TOOL_CALL/UNKNOWN),
  • track truncations/safety blocks per model,
  • correlate structured output parse failures with truncation.
Steps to reproduce
  1. Configure two chat models/providers (e.g. OpenAI-compatible + Anthropic/Gemini).
  2. Call them via ChatClient and read Generation.getMetadata().getFinishReason().
  3. Observe that the finish reason strings are provider-specific and require custom mapping to interpret consistently (e.g. stop vs end_turn vs STOP; length vs max_tokens vs MAX_TOKENS; safety-related values vary widely).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing ChatGenerationMetadata and Generation.getMetadata().getFinishReason() from ChatClient calls into the provider modules. Review how each provider supplies native finish reasons and define the additive category mapping while preserving raw values; done means known reasons are normalized and unknown values retain their raw reason and map to UNKNOWN.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spring
Domain
ai, backend-api-design
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.