perf: Cache Whisper model context between transcriptions
- Dominant language
- Rust
- Stars
- 0
- Forks
- 0
- Avg merge
- 13d 17h
- Merged PRs (30d)
- 4
Description
## What
Avoid reloading the selected Whisper model for every transcription command.
## Why
- `.planning/codebase/CONCERNS.md` notes that `transcribe_audio` creates a fresh `WhisperContext` for each run.
- Loading large models repeatedly adds latency and disk I/O.
- Model lifetime is currently scoped to a single command call.
## How
- Introduce a controlled model cache keyed by Whisper model size.
- Keep cache ownership behind a worker or state object that accounts for memory pressure.
- Evict or reload when the selected model changes.
- Add tests for model-cache selection logic without requiring real model files.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.