jellydn / jellydn/echo-note

perf: Cache Whisper model context between transcriptions

Open
#30 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
0
Forks
0
Avg merge
13d 17h
Merged PRs (30d)
4

Description

## What
Avoid reloading the selected Whisper model for every transcription command.

## Why
- `.planning/codebase/CONCERNS.md` notes that `transcribe_audio` creates a fresh `WhisperContext` for each run.
- Loading large models repeatedly adds latency and disk I/O.
- Model lifetime is currently scoped to a single command call.

## How
- Introduce a controlled model cache keyed by Whisper model size.
- Keep cache ownership behind a worker or state object that accounts for memory pressure.
- Evict or reload when the selected model changes.
- Add tests for model-cache selection logic without requiring real model files.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.