AlexsJones / AlexsJones/llmfit

[Feature]: Inference bench tui improvements

Ouverte
#909 1 commentaire 0 réactions 0 personnes assignées Voir sur GitHub
enhancement
Langage dominant
Rust
Étoiles
36.5k
Forks
2.3k
Merge moyen
3 j 3 h
PR mergées (30 j)
90

Description

### Problem or motivation

When running inference bench i can't:
1. - [ ] scroll, cursor goes out of visible models but models aren't scroll together with cursor, replicable when terminal doesn't have enough rows to display all possibly installed models
2. - [ ] disable through tui some models i don't want to inference bench
3. - [ ] pause/stop running bench test/model with keybind

Also it would be nice to have QOL:
1. - [ ] select multiple models with keybind for tests. `m` selects only one model "for comparison". (would solve issue 2 from can't stuff. as well)
2. - [ ] bench run only displayed stuff. **Example**: when having search active lets say `qwen` it lists all qwen models and according to sorting rule might list only those that are installed. Now when I will run inference bench it would automatically go only through those that were visible. (Reasoning could be I want to bench test perfect fit installed models, or only Coding, Embeding etc...)
3. - [ ] save/export/import inference bench lets say i downloaded new model and want it to compare with already done benchmarks but my human memory is short (workaround screenshot/pen-paper :D)
4. - [ ] ability to continue benchmark. I'm testing multiple models but since it takes some time and suddenly I need resources for something else i canceled it. (routing isn't displayed and I can start from zero). More or so same as issue 3.
5. - [ ] ability to directly see whats model outputting during specific test (tbf I don't understand exactly how tests work if its comparing 1 to 1 output or functionality but would be cool to see reasoning of models behind the scenes, not necessary).

Other stuff mentioned in open issues already
1. - [ ] share inference benchmarks based on hardware stats... issue 3 in QOL. could be nice solution to this. see #867

### Proposed solution

TBD

### Alternatives considered

using `--bench` cli flag, but still limited to some extent

### Feature area

TUI (new keybinding, view, or interaction)

### Would you be willing to contribute this?

No, just suggesting

### Additional context

I'm on version 1.1.10. This is so far what I was able to grasp. Overly beautiful application. If something else comes to my mind and this won't be closed I might add here regarding inference bench.

Guide de contribution

Ouvrir le guide de contribution

Piste de recherche

Start with the inference bench TUI and its existing `m` keybind for model comparison, then reproduce the scrolling and model-selection behavior in a small terminal. Review the `--bench` CLI path and related issue #867 for context. The request is broad and its proposed solution is still TBD, so a contributor would need a narrower scope and acceptance criteria before implementation.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Évaluation

Stack technique
rust
Domaine
cli
Type d'issue
Fonctionnalité
Difficulté
5/5
Temps estimé
Plus d'une semaine
Activité
Active
Clarté
À clarifier
Accessibilité débutants
25/100

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.