microsoft / microsoft/onnxruntime-genai
onnxruntime-genai - QNN Memory Usage
Open
performance
- Dominant language
- C++
- Stars
- 1.1k
- Forks
- 354
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 85
Description
Referring to https://github.com/microsoft/onnxruntime-genai/issues/961, will it address the memory aspect of running big models on the npu?
thanks
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading the referenced issue #961 and its discussion. Determine whether that work explicitly covers memory usage when running large models on an NPU; the current issue does not define a change, reproduction, target memory behavior, or acceptance criteria, so its scope must be clarified before implementation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- ai, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 15/100