AlexsJones / AlexsJones/llmfit
[Feature]: Inference bench tui improvements
- Ngôn ngữ chính
- Rust
- Star
- 36.3k
- Fork
- 2.3k
- Merge trung bình
- 2 ngày 18 giờ
- Pull request đã merge (30 ngày)
- 85
Mô tả
### Problem or motivation
When running inference bench i can't:
1. - [ ] scroll, cursor goes out of visible models but models aren't scroll together with cursor, replicable when terminal doesn't have enough rows to display all possibly installed models
2. - [ ] disable through tui some models i don't want to inference bench
3. - [ ] pause/stop running bench test/model with keybind
Also it would be nice to have QOL:
1. - [ ] select multiple models with keybind for tests. `m` selects only one model "for comparison". (would solve issue 2 from can't stuff. as well)
2. - [ ] bench run only displayed stuff. **Example**: when having search active lets say `qwen` it lists all qwen models and according to sorting rule might list only those that are installed. Now when I will run inference bench it would automatically go only through those that were visible. (Reasoning could be I want to bench test perfect fit installed models, or only Coding, Embeding etc...)
3. - [ ] save/export/import inference bench lets say i downloaded new model and want it to compare with already done benchmarks but my human memory is short (workaround screenshot/pen-paper :D)
4. - [ ] ability to continue benchmark. I'm testing multiple models but since it takes some time and suddenly I need resources for something else i canceled it. (routing isn't displayed and I can start from zero). More or so same as issue 3.
5. - [ ] ability to directly see whats model outputting during specific test (tbf I don't understand exactly how tests work if its comparing 1 to 1 output or functionality but would be cool to see reasoning of models behind the scenes, not necessary).
Other stuff mentioned in open issues already
1. - [ ] share inference benchmarks based on hardware stats... issue 3 in QOL. could be nice solution to this. see #867
### Proposed solution
TBD
### Alternatives considered
using `--bench` cli flag, but still limited to some extent
### Feature area
TUI (new keybinding, view, or interaction)
### Would you be willing to contribute this?
No, just suggesting
### Additional context
I'm on version 1.1.10. This is so far what I was able to grasp. Overly beautiful application. If something else comes to my mind and this won't be closed I might add here regarding inference bench.
Hướng dẫn đóng góp
Đánh giá
Issue này chưa được đánh giá.