open-webui / open-webui/computer
bug: Every new request invalidates the full context cache, causing llama.cpp to reprocess the entire context
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 569
- Forks
- 79
- PR merge metrics
- No merged PRs in 30d
Description
Hello,
I noticed that every follow-up request in an ongoing chat appears to invalidate the complete context cache, forcing llama.cpp to reprocess the entire prompt from scratch even though the chat history is not modified.
Steps to reproduce:
- Start a chat session in Open Computer connected to a llama.cpp backend (in my case: model
Qwen3.8 27B (xhigh)proxied on port 45723). - Send a message and wait for the response to complete (the full context is loaded and cached).
- Send a follow-up message without modifying the chat history.
- Check the llama.cpp logs for the second request.
Instead of reusing the cached KV context and only processing the new tokens, the logs show the full prompt being reprocessed from the start on the follow-up request (the prompt is roughly 100k+ tokens based on the progress fractions, so the re-prefill penalty is substantial):
218.52.490.688 I srv proxy_reques: proxying request to model Qwen3.8 27B (xhigh) on port 45723
[45723] 202.49.229.491 I slot get_availabl: id 1 | task -1 | selected slot by LRU, t_last = 13320538860
[45723] 202.49.232.130 I slot launch_slot_: id 1 | task 129052 | processing task, is_child = 0
[45723] 202.54.370.411 I slot print_timing: id 1 | task 129052 | prompt processing, n_tokens = 4096, progress = 0.04, t = 4.92 s / 832.76 tokens per second
[45723] 202.56.400.910 I slot print_timing: id 1 | task 129052 | prompt processing, n_tokens = 5590, progress = 0.05, t = 6.95 s / 804.42 tokens per second
[45723] 202.59.256.537 I slot print_timing: id 1 | task 129052 | prompt processing, n_tokens = 7638, progress = 0.07, t = 9.80 s / 779.01 tokens per second
[45723] 203.02.166.965 I slot print_timing: id 1 | task 129052 | prompt processing, n_tokens = 9686, progress = 0.09, t = 12.72 s / 761.77 tokens per second
[45723] 203.05.183.430 I slot print_timing: id 1 | task 129052 | prompt processing, n_tokens = 11734, progress = 0.10, t = 15.73 s / 745.89 tokens per second
[45723] 203.08.304.425 I slot print_timing: id 1 | task 129052 | prompt processing, n_tokens = 13782, progress = 0.12, t = 18.85 s / 731.04 tokens per second
Expected behavior:
For consecutive requests within the same chat where the history is unchanged, the cached context should be preserved and only the newly added tokens should be processed.
Thanks in advance!
Joshua Krimmer
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the request path that proxies follow-up chat requests to the llama.cpp backend, then compare how chat history and context state are constructed between consecutive requests. Use the provided llama.cpp timing logs to verify the current full reprocessing, and consider the issue done when unchanged history reuses the cached context and processes only new tokens.
Written by the indexing model from the issue text.
Assessment
- Domain
- backend, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100