RAG causes memory loss in large models

Expected Behavior

rag should not affect the memory function of the large model, so that my question is inaccurate

Current Behavior

After implementing rag, the memory function of the large model is weakened or even lost, but I did turn on the chat memory function, which is normal in interfaces without rag

Context

For example, I turned on 10 memories, after asking three or four sentences, after asking it to list the questions I asked before, it didn't know at all, if you go directly to the big model Q&A, don't go to rag, there is no problem with memory, I guess it is that every request will retrieve the vector library and take out the data to stuff the large model, too much content leads to the weakening of memory, but I don't know if this is the case. Has anyone ever encountered this situation?

Start by reproducing the reported behavior with RAG and chat memory enabled, using the example of asking the model to recall earlier questions. Compare the result with direct large-model Q&A without RAG; done means identifying the cause and preserving memory while RAG is enabled.