SciSharp / SciSharp/LLamaSharp
TODO: implementing the manual history tracking fix in StatefulExecutorBase
Nobody has claimed this yet.
- Dominant language
- C#
- Stars
- 3.8k
- Forks
- 508
- Avg merge
- 1d 5h
- Merged PRs (30d)
- 12
Description
Description
PR: https://github.com/SciSharp/LLamaSharp/pull/1373
Explanation: I am holding off on implementing the manual history tracking fix in this PR for StatefulExecutorBase. Rebuilding the KV cache manually for models that don't support native memory shifting is highly complex and risks silent state corruption, especially when dealing with session caching and multimodal inputs. Keeping the ContextOverflowException guard as-is forces the calling application to handle the overflow gracefully (e.g., starting a new chat) rather than risking a desynced model state. We can revisit this if we see high user impact, but stability comes first.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names StatefulExecutorBase and points to PR 1373; start by reading that PR and the current ContextOverflowException guard. Done would require an agreed, safe approach for manual KV-cache history tracking that preserves session caching and multimodal state without silent corruption.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- ai, backend
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100