microsoft / microsoft/onnxruntime-genai
Memory leak/growth in iOS build when pre-filling tokens via generator's `AppendTokenSequences` API
@baijumeswani is already working on this.
Since Oct 3, 2025.
- Dominant language
- C++
- Stars
- 1.1k
- Forks
- 354
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 85
Description
Describe the bug
We are noticing a memory leak/growth in the iOS build when we load historical chats via the generator's AppendTokenSequences API.
Note that the same code works on the Android side.
Here's the snippet of wrapper code in C++ we have in common to Android/iOS:
void ONNXLLMExecutor::add_input_to_generator(const std::string& input) {
auto sequences = OgaSequences::Create();
_tokenizer->Encode(input.c_str(), *sequences);
_generator->AppendTokenSequences(*sequences);
}
Expected behavior
No memory leak/growth when tokens are pre-filled from a historical chat
Screenshots
N/A
Desktop (please complete the following information):
- OS: [e.g. iOS]
- Browser [e.g. chrome, safari]
- Version [e.g. 22]
Smartphone (please complete the following information):
- Device: [e.g. iPhone 15]
- OS: [e.g. iOS 18.5, 17.5]
- Version [v0.7.0 & v0.8.2]
Additional context
Possibly related issue https://github.com/microsoft/onnxruntime-genai/issues/1075 which was auto closed.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.