bug: problematic cache reuse

Bug

On moderate to long conversation, I noticed that the inference engine cache is not being used efficiently .
Turns becomes longer and longer, and backend metrics and logs confirms it.

Reproduction

Run a long multi-turn conversation and measure speed + check you backend logs, notice mostly full prompt reprocessing at every turn, including in between tool calls.

Likely Root cause

In prompt_templates, {{MEMORY}} and maybe others fields like {{FILE_TREE}} are recomputed and alterated at each turns.
It modifies the conversation history, and the backend is loosing most of its cache because of the early divergence.

Suggestions

These fields should probably be frozen for the current discussion, and if the Agent needs update:

• they can use tool call to retrieve updated values
• and / or the harness can inject updated value at the bottom of the conversation rather than alterating the prefix

Environment

• cptr 0.9.13
• llamacpp

_Disclosure: this issue was investigated and root-caused with the help of ChatGPT , and is written and submitted on my behalf at my request.

Start by locating prompt_templates handling of {{MEMORY}} and {{FILE_TREE}}, then run a long multi-turn conversation with tool calls while checking backend metrics and logs. Done means the conversation prefix no longer changes unnecessarily between turns and cache reuse improves without losing required updates.