spring-projects / spring-projects/spring-ai

JdbcChatMemoryRepository do not persist Media in UserMessage

Open
#3,597 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

status: to-discuss
Dominant language
Java
Stars
9.5k
Forks
2.9k
Avg merge
1d 7h
Merged PRs (30d)
6

Description

Expected Behavior
When using JdbcChatMemoryRepository it is expected that you will be able to continue a chat conversation keeping all the information that the User passed to the LLM model such as the text and also the Media resources (i.e.: for multimodal vision capable LLM models).

Current Behavior
JdbcChatMemoryRepository, as it is designed in version 1.0.0, persists only the following information:

  • conversation_id (not an intrinsic field of the Message, passed in the saveAll method to group the messages pertaining to a given conversation)
  • content (the text part of the message)
  • type (the message type as a string representation of the Enum org.springframework.ai.chat.messages.MessageType)
  • timestamp (a database generated field, for ordered retrieval purpose)

This behaviour lack a way to persist the Media attached to a UserMessage.

Context
With the use of JdbcChatMemoryRepository the LLM lose the capability to continue the conversation about a previously attached Media.

InMemoryChatMemory does not have any issues about keeping the Media attached to the Chat History.

I understand that the ChatMemory is not to be intended as a strategy to persist a Chat History but keeping Media attached to the UserMessage for the purpose of continuing a conversation about a given context (Media included) should be the standard behaviour of every ChatMemory implementation.

The only workaround I found is to write a custom implementation of ChatMemoryRepository capable to persist and retrieve the chat memory (at least for the UserMessage and AssistantMessage message types) serializing the content of the Media field if present.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the JdbcChatMemoryRepository entry point and inspect how UserMessage content, type, and timestamp are persisted and reconstructed. Trace the database representation used for chat memory and determine how attached Media is handled during save and retrieval. Done means a resumed conversation preserves UserMessage Media as well as text, with coverage for the affected message flow.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spring
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.