bytechefhq / bytechefhq/bytechef
[task] Spring AI Improvements
- Dominant language
- Java
- Stars
- 1k
- Forks
- 170
- Avg merge
- 11h 25m
- Merged PRs (30d)
- 115
Description
## Spring AI Blog Analysis - Missing Functionality in ByteChef
> **Current ByteChef State**: Spring AI 2.0.0-M4, 12 LLM providers, 14 vector stores, MCP server (streamable HTTP), 8 chat memory backends, guardrails advisor, basic agent evaluation, structured output, tool calling via ToolCallback/FunctionToolCallback.
---
- [ ] https://spring.io/blog/2025/11/12/spring-ai-1-1-GA-released
**Missing functionality:**
- [ ] Prompt caching support for Anthropic Claude (5 cache strategies: NONE, SYSTEM_ONLY, TOOLS_ONLY, SYSTEM_AND_TOOLS, CONVERSATION_HISTORY)
- [ ] Prompt caching support for AWS Bedrock (Claude + Nova models)
- [ ] Recursive Advisors - advisors that call other advisors in chains for multi-step AI workflows (self-improving agents, LLM-as-a-Judge)
- [ ] Reasoning/Thinking Mode support across providers (Anthropic streaming thinking events, OpenAI reasoning content, Ollama thinking mode)
- [ ] ReasoningContent API - inspect and utilize model's reasoning process
- [ ] Google GenAI SDK integration (direct API key auth, separate from Vertex AI Gemini)
- [ ] ElevenLabs Text-to-Speech integration with streaming audio generation
- [ ] TextToSpeechModel interface - consistent API across TTS providers
- [ ] Anthropic Citations API for source attribution and traceability
- [ ] Anthropic Tool Choice support (Auto, Any, Tool, None modes)
- [ ] Mistral AI OCR API integration for document/image text extraction
- [ ] OpenSearch Approximate k-NN for performance-optimized vector search
- [ ] VectorStoreRetriever interface - read-only abstraction for vector stores
- [ ] Observation context enhancements (Micrometer) for better monitoring
- [ ] MarkdownDocumentReader with batch processing for knowledge base
- [ ] Enhanced multimodal PDF support for document processing
---
- [ ] https://spring.io/blog/2025/12/05/spring-ai-1-1-1-available-now
**Missing functionality:**
- [ ] OpenAI Java SDK native integration for improved type safety and API coverage
- [ ] ChatClient native structured output support
- [ ] Claude Skills API with Files API integration (document generation)
- [ ] Google Gemini ThinkingLevel configuration in ThinkingConfig
- [ ] Google Gemini Safety Ratings in response metadata
- [ ] Gemini 3 Pro thought signatures support for function calling
- [ ] Vector Store ISNULL/ISNOTNULL filter expressions for null value checks
- [ ] ToolCallAdvisor extensibility through hook methods
---
- [ ] https://spring.io/blog/2025/11/04/spring-ai-recursive-advisors/
**Note:** Blog URL redirects to prompt caching article. Recursive advisors content covered under 1.1 GA blog above.
**Missing functionality:**
- [ ] Recursive Advisor framework - configurable chain depth, observation support
- [ ] Built-in recursive advisor implementations for iterative output refinement
- [ ] LLM-as-a-Judge evaluation via recursive advisors
---
- [ ] https://spring.io/blog/2025/10/27/spring-ai-anthropic-prompt-caching-blog/
**Missing functionality:**
- [ ] AnthropicCacheStrategy enum with 5 strategies (NONE, SYSTEM_ONLY, TOOLS_ONLY, SYSTEM_AND_TOOLS, CONVERSATION_HISTORY)
- [ ] Cache TTL options (5-minute default, 1-hour extended via AnthropicCacheTtl.ONE_HOUR)
- [ ] Automatic cache breakpoint management (max 4 breakpoints per request)
- [ ] CacheEligibilityResolver for message/tool cache qualification
- [ ] CacheBreakpointTracker with thread-safe tracking per request
- [ ] Cache metrics via response metadata (cacheCreationInputTokens, cacheReadInputTokens)
- [ ] Content length filtering per message type with custom token counting
- [ ] Streaming support for prompt caching
- [ ] Conversation history incremental caching with aggregate eligibility checking
---
- [ ] https://spring.io/blog/2025/10/30/spring-ai-bedrock-prompt-caching-blog/
**Missing functionality:**
- [ ] AWS Bedrock prompt caching for Claude models (Opus 4.1, Opus 4, Sonnet 4.5, Sonnet 4, Haiku 4.5, 3.7 Sonnet, 3.5 Haiku)
- [ ] AWS Bedrock prompt caching for Amazon Nova models (Micro, Lite, Pro, Premier) - system and conversation only
- [ ] BedrockCacheStrategy with model-specific constraints (no tool caching on Nova)
- [ ] Bedrock cache metrics (cacheWriteInputTokens, cacheReadInputTokens)
- [ ] Model-specific minimum token thresholds per cache checkpoint
---
- [ ] https://spring.io/blog/2025/12/11/spring-ai-tool-search-tools-tzolov
**Missing functionality:**
- [ ] ToolSearchToolCallAdvisor for on-demand tool discovery (34-64% token savings)
- [ ] VectorToolSearcher - semantic search using vector embeddings for fuzzy tool matching
- [ ] LuceneToolSearcher - keyword search using Apache Lucene for exact term matching
- [ ] RegexToolSearcher - pattern-based tool matching
- [ ] Dynamic tool context expansion - only load relevant tool definitions into LLM context
- [ ] Integration with Spring AI Recursive Advisors for multi-turn discovery
---
- [ ] https://spring.io/blog/2025/12/11/spring-ai-2-0-0-M1-available-now
**Missing functionality:**
- [ ] Redis Vector Store text search capabilities and range queries
- [ ] Redis Vector Store HNSW index parameter tuning (M, efConstruction, efRuntime)
- [ ] Anthropic Citations API support (reference specific document parts in responses)
- [ ] Claude Skills with Files API for document generation (Excel, PowerPoint, Word, PDF)
- [ ] Claude Tool Choice support (Auto, Any, Tool, None modes)
---
- [ ] https://spring.io/blog/2026/01/23/spring-ai-2-0-0-M2-available-now
**Missing functionality:**
- [ ] Amazon S3 vector store backend support
- [ ] Infinispan vector store integration
- [ ] Amazon Bedrock Knowledge Base vector store support
- [ ] Redis semantic cache advisor implementation
- [ ] Dynamic tool schema enhancement support
- [ ] JSON schema customization for structured output generation
- [ ] Ollama embeddings configurable dimensions parameter
- [ ] Mistral AI native structured output using JSON schema
- [ ] McpSyncServerCustomizer/McpAsyncServerCustomizer for MCP auto-configuration
- [ ] ToolCallAdvisor conversationHistoryEnabled option
---
- [ ] https://spring.io/blog/2026/03/17/spring-ai-2-0-0-M3-and-1-1-3-and-1-0-4-available
**Missing functionality:**
- [x] Migrate Anthropic integration to official Anthropic Java SDK (replaces REST/WebClient-based implementation)
- [x] Jackson 3 migration - update all AI module code from `com.fasterxml.jackson` to `tools.jackson` package
- [x] Update MCP annotation imports from `org.springaicommunity.mcp` to `org.springframework.ai.mcp.annotation` (now in Spring AI core)
- [x] Update MCP transport imports from `io.modelcontextprotocol.sdk` to `org.springframework.ai.mcp` (relocated to Spring AI)
- [x] Consolidate MCP client customizers - replace `McpAsyncClientCustomizer`/`McpSyncClientCustomizer` with unified `McpClientCustomizer`
- [ ] ToolCallAdvisor streaming response support - enable real-time tool call execution during streaming
- [ ] Add Claude 4.6 model support (and remove deprecated Claude 3 Opus/Sonnet/Haiku references)
- [ ] Native structured output support for BedrockProxyChatModel
- [ ] Filter expression converter improvements across vector store implementations
- [ ] Docker Compose support for Milvus vector database
- [ ] System prompt isolation in semantic cache
- [ ] Enhanced TokenTextSplitter with better EncodingType support
- [ ] Custom embedding dimensions via properties for OpenAiEmbeddingModel
- [ ] Anthropic SIGNATURE_DELTA streaming event support
- [ ] Model-specific thinking levels with Google GenAI SDK 1.37.0
- [ ] Handle HuggingFace removal from Spring AI core (evaluate community alternative or remove)
- [ ] Remove conversation history from ToolContext (breaking change) and update dependent code
- [ ] Update ChatOptions builders to new generic builder patterns (DeepSeek, Bedrock, MistralAi, AzureOpenAi, Anthropic)
- [ ] CVE-2026-22729 and CVE-2026-22730 security fixes - verify dependencies are updated
---
- [ ] https://spring.io/blog/2026/03/26/spring-ai-2-0-0-M4-and-1-1-4-and-1-0-5-available
**Missing functionality:**
- [ ] Google Search alongside custom tools in Gemini 3.x models
- [ ] Dynamic disabling of native structured output functionality
- [ ] Handle Vertex AI Gemini deprecation - plan migration to Google GenAI SDK
- [ ] Handle ZhiPu AI deprecation - evaluate impact and removal plan
- [ ] Handle OCI GenAI deprecation - evaluate impact and removal plan
- [ ] Update to Anthropic SDK 2.17.0 (from M4 dependency upgrades)
- [ ] Update to OpenAI SDK 4.28.0 (from M4 dependency upgrades)
- [ ] Update to Google Generative AI SDK 1.44.0 (from M4 dependency upgrades)
- [ ] Fix Redis vector store string value handling for TAG/TEXT filter values
- [ ] Fix Neo4j vector store key handling in filter expression converter
- [ ] Fix filter expression text parser identifier edge cases
---
- [ ] https://github.com/spring-ai-community/spring-ai-replicate
**Missing functionality:**
- [ ] Replicate.com integration - access to hundreds of hosted AI models
- [ ] ReplicateChatModel for conversational interactions
- [ ] ReplicateMediaModel for image/video/audio generation
- [ ] ReplicateStringModel for classification/detection tasks
- [ ] ReplicateStructuredModel for embeddings/JSON responses
- [ ] Synchronous and streaming (reactive Flux) request modes for Replicate
---
- [ ] https://spring.io/blog/2025/10/28/agents-and-benchmarks
**Missing functionality:**
- [ ] Spring AI Agents - AgentClient abstraction for CLI-based agent invocation (Claude Code, Gemini CLI, Amazon Q, Amp, Codex)
- [ ] Agent primitives: Goals, Context, Tools, Judges, Sandbox abstractions
- [ ] Agent sandbox support (local + Docker container execution)
- [ ] Spring AI Bench - benchmarking suite for Java-centric AI agent evaluation
- [ ] Enterprise development task benchmarks (issue triage, PR review, test coverage uplift, compliance validation)
---
- [ ] https://spring.io/blog/2025/12/23/spring-ai-tool-argument-augmenter-tzolov
**Missing functionality:**
- [ ] AugmentedToolCallbackProvider for dynamic tool input schema augmentation
- [ ] Capture LLM reasoning/inner thoughts during tool calls
- [ ] Chain of Thought reasoning capture (reasoningSteps, alternatives, conclusion)
- [ ] Tool usage analytics (toolSelectionReason, expectedOutcome, userIntent)
- [ ] Multi-agent coordination metadata via augmented arguments (agentId, notifyAgents, sharedContext)
- [ ] MCP-compatible augmentation (works with both @Tool and @McpTool)
---
- [ ] https://spring.io/blog/2026/01/13/spring-ai-generic-agent-skills
**Missing functionality:**
- [ ] SkillsTool - modular skill discovery and loading with progressive disclosure (name+description at startup, full content on demand)
- [ ] Skill definitions as Markdown files with YAML frontmatter (SKILL.md)
- [ ] FileSystemTools integration for reading skill-bundled reference files
- [ ] ShellTools integration for executing skill-bundled helper scripts
- [ ] Classpath and file system skill directory scanning
- [ ] Claude Code Skills compatibility
---
- [ ] https://spring.io/blog/2026/01/16/spring-ai-ask-user-question-tool
**Missing functionality:**
- [ ] AskUserQuestionTool - enable agents to ask users multiple-choice questions during execution
- [ ] QuestionHandler interface with console-based and web-based (WebSocket/SSE + REST) implementations
- [ ] Single-select and multi-select question support with free-text input
- [ ] CompletableFuture-based async UI bridge for web question handlers
- [ ] Integration with MCP Elicitation (@McpElicitation annotation)
---
- [ ] https://spring.io/blog/2026/01/20/spring-ai-agentic-patterns-3-todowrite
**Missing functionality:**
- [ ] TodoWriteTool - task decomposition and tracking during LLM execution (prevents "lost in the middle" failures)
- [ ] Todo item lifecycle (pending → in_progress → completed) with single in_progress constraint
- [ ] Real-time progress tracking with visual indicators and percentage
- [ ] Event-driven progress updates via Spring ApplicationEvents (TodoUpdateEvent)
- [ ] Self-governing complexity assessment (LLM decides when task tracking is needed)
- [ ] Integration with Chat Memory and ToolCallAdvisor
---
- [ ] https://spring.io/blog/2026/01/27/spring-ai-agentic-patterns-4-task-subagents
**Missing functionality:**
- [ ] TaskTool for hierarchical agent architectures with subagent orchestration
- [ ] Agent Registry - catalog of subagent names and descriptions for LLM-driven delegation
- [ ] Subagent configuration via Markdown files with YAML frontmatter (name, description, tools, model)
- [ ] Multi-model routing - route tasks to different LLMs based on complexity (haiku/sonnet/opus tiers)
- [ ] Isolated context windows per subagent preventing context pollution
- [ ] Background task execution with TaskOutputTool for async result retrieval
- [ ] Built-in subagent types (Explore, General-Purpose, Plan, Bash)
- [ ] Parallel subagent execution
---
- [ ] https://spring.io/blog/2026/01/29/spring-ai-agentic-patterns-a2a-integration
**Missing functionality:**
- [ ] Agent2Agent (A2A) protocol support - open standard for AI agent communication
- [ ] A2A Server - expose Spring AI agents as A2A-compliant servers with auto-configured endpoints
- [ ] A2A Client - discover and communicate with remote A2A agents
- [ ] AgentCard discovery at `/.well-known/agent-card.json` with skills, capabilities metadata
- [ ] Multi-agent orchestration - host agent discovers and routes to specialized remote agents
- [ ] LLM-driven routing - model decides which specialized agents to invoke
- [ ] Spring AI @Tool as A2A client bridge for seamless remote agent invocation
---
- [ ] https://spring.io/blog/2026/04/07/spring-ai-agentic-patterns-6-memory-tools
**Missing functionality:**
- [ ] AutoMemoryTools — sandboxed memory operations implementing Claude API Memory Tool spec (MemoryView, MemoryCreate, MemoryStrReplace, MemoryInsert, MemoryDelete, MemoryRename)
- [ ] AutoMemoryToolsAdvisor — zero-boilerplate advisor with automatic system prompt injection, tool deduplication, and configurable memory consolidation trigger predicate
- [ ] Advisor chain ordering — runs first, before ToolCallAdvisor and ChatMemoryAdvisor
- [ ] Filesystem-based memory integration (Option C) — FileSystemTools + ShellTools mapping Read/Write/Edit to memory operations (no sandbox, by convention)
- [ ] MEMORY.md index file pattern — flat list of one-line pointers, always loaded at session start
- [ ] Memory file format — Markdown with YAML frontmatter (name, description, type)
- [ ] Four typed memory categories: user, feedback, project, reference (with distinct save/usage semantics)
- [ ] Built-in system prompt templates (AUTO_MEMORY_TOOLS_SYSTEM_PROMPT.md, AUTO_MEMORY_FILESYSTEM_TOOLS_SYSTEM_PROMPT.md)
- [ ] Path sandboxing to configured memories root directory (.memoriesRootDirectory)
- [ ] Memory consolidation trigger predicate `(request, Instant) → boolean` — time-elapsed, utterance matching, probabilistic, turn-count strategies
- [ ] spring-ai-agent-utils dependency integration (`org.springaicommunity:spring-ai-agent-utils:0.7.0`, requires Spring AI 2.0.0-M4+)
- [ ] Multi-provider compatibility (Claude / Gemini / OpenAI) across all three integration options
- [ ] Note: complements (not replaces) ChatMemory — window-based context vs curated persistent facts. Evaluate positioning alongside existing `ai-agent-skill` module (which stores zipped skill archives, not markdown+frontmatter files)
---
- [ ] https://spring.io/blog/2026/04/15/spring-ai-session-management
**Missing functionality:**
- [ ] **⚠️ Migration planning: ChatMemory → Session API** — Spring AI 2.1 (Nov 2026) deprecates `ChatMemory`; all 8 ByteChef chat-memory backends (JDBC, Redis, in-memory, MongoDB, Neo4j, Cassandra, CosmosDB, VectorStore) need migration paths to the new Session API
- [ ] `Session` immutable value object — id, userId, TTL, arbitrary metadata
- [ ] `SessionEvent` model — wraps `Message` with UUID, sessionId, timestamp, optional branch label, framework flags (e.g., `METADATA_SYNTHETIC`)
- [ ] `SessionService` interface + `DefaultSessionService` implementation for session lifecycle operations
- [ ] `SessionRepository` abstraction (replaces `ChatMemoryRepository`)
- [ ] `InMemorySessionRepository` for development
- [ ] `Turn` abstraction — atomic conversation unit (one `UserMessage` + subsequent events until next `UserMessage`)
- [ ] Compaction triggers: `TurnCountTrigger`, `TokenCountTrigger`, `CompositeCompactionTrigger` (OR-composite)
- [ ] Compaction strategies: `SlidingWindowCompactionStrategy`, `TurnWindowCompactionStrategy`, `TokenCountCompactionStrategy`, `RecursiveSummarizationCompactionStrategy`
- [ ] Cut-point snapping to turn boundaries (prevents orphaned tool results during compaction)
- [ ] `SessionMemoryAdvisor` — transparent ChatClient pipeline integration (load history, prepend to prompt, append new messages, run compaction)
- [ ] `SESSION_ID_CONTEXT_KEY` — per-call session ID passing with auto-creation fallback
- [ ] Multi-agent branch support — `SessionEvent.branch` as dot-separated path (e.g., `"orch.researcher"`); null-branch events visible to all agents; synthetic summaries always null-branch
- [ ] `EventFilter.forBranch()` — branch isolation for multi-agent orchestration (agent sees null-branch + own branch + ancestors)
- [ ] `SessionEventTools` implementing MemGPT Recall Storage pattern
- [ ] `conversation_search` tool (auto-discovered) — keyword search with pagination over full verbatim event log (searchable even after compaction pruning)
- [ ] `spring-ai-session-jdbc` backend — append-only `AI_SESSION` / `AI_SESSION_EVENT` schema, supports PostgreSQL / MySQL / MariaDB / H2
- [ ] `spring.ai.session.repository.jdbc.initialize-schema` auto-configuration property
- [ ] Optimistic CAS writes across all repository implementations
- [ ] `CreateSessionRequest` builder with `.userId()`
- [ ] `service.getMessages(sessionId)` — returns `List` ready for LLM
- [ ] Build integration: `spring-ai-session-bom` (0.2.0), `spring-ai-starter-session-jdbc` (production), `spring-ai-session-management` (dev) starters
---
- [ ] https://springaicommunity.mintlify.app/projects
1. https://springaicommunity.mintlify.app/projects/incubating/spring-agui
**Note:** ByteChef already has basic AGUI integration.
- [ ] Review and enhance current AGUI integration for full AG-UI protocol compliance
- [ ] Multi-modal agent-UI communication (text, images, other media)
- [ ] Agent orchestration through shared interface state
2. https://springaicommunity.mintlify.app/projects/incubating/spring-ai-playground
**Missing functionality:**
- [ ] Self-hosted AI playground with Chat, VectorDB, and MCP testing capabilities
- [ ] VectorDB playground for RAG workflow testing (upload, chunk, embed, search)
- [ ] MCP playground for registering and testing MCP tools across transport types
3. https://springaicommunity.mintlify.app/projects/incubating/mcp-security
**Missing functionality:**
- [ ] Enterprise MCP security framework (TLS/SSL, OAuth2/OIDC, RBAC)
- [ ] MCP audit logging for security event tracking and compliance
- [ ] Input validation with schema enforcement and rate limiting for MCP servers
- [ ] Spring Security integration for MCP
4. https://springaicommunity.mintlify.app/projects/incubating/agent-sandbox
**Missing functionality:**
- [ ] Unified agent sandbox API with multiple backends (Local, Docker, E2B cloud microVMs)
- [ ] Container-based isolation using Testcontainers (DockerSandbox)
- [ ] E2B Firecracker microVMs with MCP support for maximum isolation
- [ ] Fluent file operations API for sandbox file management
- [ ] Session persistence and sandbox reconnection
5. https://springaicommunity.mintlify.app/projects/incubating/agent-judge
**Note:** ByteChef has basic evaluation (ContainsTextJudge). Needs expansion.
- [ ] Deterministic judges - file existence/content validation, pattern matching, custom rules
- [ ] Command judges - shell command execution with exit code validation, build tool integration
- [ ] LLM-powered judges with customizable prompts for AI-based correctness assessment
- [ ] Jury system - multi-judge aggregation with configurable voting strategies (majority, consensus, weighted)
- [ ] Three score types: boolean (pass/fail), numerical (0-100), categorical
- [ ] Composable judges via logical operators (AND, OR)
6. https://springaicommunity.mintlify.app/projects/incubating/agent-client
**Missing functionality:**
- [ ] Unified Agent Client framework for CLI-based AI agents (Claude Code, Gemini, Amazon Q, Amp, Codex)
- [ ] Five core abstractions: Goals, Tools, Context, Judges, Sandboxes
- [ ] Agent Advisors pattern (mirrors Spring AI ChatClient design)
- [ ] Portable MCP server definitions translating to each provider's native format
- [ ] Post-execution metrics and observability hooks
Contributor guide
Assessment
This issue has not been assessed yet.