AlexsJones / AlexsJones/llmfit
[Feature]: Inference bench tui improvements
- Langage dominant
- Rust
- Étoiles
- 36.5k
- Forks
- 2.3k
- Merge moyen
- 3 j 3 h
- PR mergées (30 j)
- 90
Description
### Problem or motivation
When running inference bench i can't:
1. - [ ] scroll, cursor goes out of visible models but models aren't scroll together with cursor, replicable when terminal doesn't have enough rows to display all possibly installed models
2. - [ ] disable through tui some models i don't want to inference bench
3. - [ ] pause/stop running bench test/model with keybind
Also it would be nice to have QOL:
1. - [ ] select multiple models with keybind for tests. `m` selects only one model "for comparison". (would solve issue 2 from can't stuff. as well)
2. - [ ] bench run only displayed stuff. **Example**: when having search active lets say `qwen` it lists all qwen models and according to sorting rule might list only those that are installed. Now when I will run inference bench it would automatically go only through those that were visible. (Reasoning could be I want to bench test perfect fit installed models, or only Coding, Embeding etc...)
3. - [ ] save/export/import inference bench lets say i downloaded new model and want it to compare with already done benchmarks but my human memory is short (workaround screenshot/pen-paper :D)
4. - [ ] ability to continue benchmark. I'm testing multiple models but since it takes some time and suddenly I need resources for something else i canceled it. (routing isn't displayed and I can start from zero). More or so same as issue 3.
5. - [ ] ability to directly see whats model outputting during specific test (tbf I don't understand exactly how tests work if its comparing 1 to 1 output or functionality but would be cool to see reasoning of models behind the scenes, not necessary).
Other stuff mentioned in open issues already
1. - [ ] share inference benchmarks based on hardware stats... issue 3 in QOL. could be nice solution to this. see #867
### Proposed solution
TBD
### Alternatives considered
using `--bench` cli flag, but still limited to some extent
### Feature area
TUI (new keybinding, view, or interaction)
### Would you be willing to contribute this?
No, just suggesting
### Additional context
I'm on version 1.1.10. This is so far what I was able to grasp. Overly beautiful application. If something else comes to my mind and this won't be closed I might add here regarding inference bench.
Guide de contribution
Ouvrir le guide de contribution
Piste de recherche
Start with the inference bench TUI and its existing `m` keybind for model comparison, then reproduce the scrolling and model-selection behavior in a small terminal. Review the `--bench` CLI path and related issue #867 for context. The request is broad and its proposed solution is still TBD, so a contributor would need a narrower scope and acceptance criteria before implementation.
Rédigé par le modèle d'indexation à partir du texte de l'issue.
Évaluation
- Stack technique
- rust
- Domaine
- cli
- Type d'issue
- Fonctionnalité
- Difficulté
- 5/5
- Temps estimé
- Plus d'une semaine
- Activité
- Active
- Clarté
- À clarifier
- Accessibilité débutants
- 25/100