microsoft / microsoft/ai-dev-gallery
[FEATURE] Add ability to stop a sample from running if not enough resources
Nobody has claimed this yet.
- Dominant language
- C#
- Stars
- 1.5k
- Forks
- 223
- PR merge metrics
- No merged PRs in 30d
Description
Describe the bug
When app is open, a model is always loaded. Hence GPU/CPU is always in use. If user is using a large model, it would contribute to low performance of computer and memory leak. Need a way to eject the model when not using the playground.
To Reproduce
Steps to reproduce the behavior:
- Download Phi3 medium
- Try run generation sample
- Have task manager open to monitor GPU usage
- 16GB is barely enough and no way to eject the model to stop GPU usage
Expected behavior
A button to eject the model to free processors.
Originally created by @beth-panx
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the playground's model-loading flow and reproduce the Phi3 medium generation sample while monitoring GPU and memory usage in Task Manager. Done means the playground provides a way to eject the loaded model when it is not in use, releasing its CPU and GPU resources.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- ai, performance
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100