AlexsJones / AlexsJones/llmfit

[Feature]: Inference bench tui improvements

Đang mở
#909 1 bình luận 0 reaction 0 người được giao Xem trên GitHub
enhancement
Ngôn ngữ chính
Rust
Star
36.3k
Fork
2.3k
Merge trung bình
2 ngày 18 giờ
Pull request đã merge (30 ngày)
85

Mô tả

### Problem or motivation

When running inference bench i can't:
1. - [ ] scroll, cursor goes out of visible models but models aren't scroll together with cursor, replicable when terminal doesn't have enough rows to display all possibly installed models
2. - [ ] disable through tui some models i don't want to inference bench
3. - [ ] pause/stop running bench test/model with keybind

Also it would be nice to have QOL:
1. - [ ] select multiple models with keybind for tests. `m` selects only one model "for comparison". (would solve issue 2 from can't stuff. as well)
2. - [ ] bench run only displayed stuff. **Example**: when having search active lets say `qwen` it lists all qwen models and according to sorting rule might list only those that are installed. Now when I will run inference bench it would automatically go only through those that were visible. (Reasoning could be I want to bench test perfect fit installed models, or only Coding, Embeding etc...)
3. - [ ] save/export/import inference bench lets say i downloaded new model and want it to compare with already done benchmarks but my human memory is short (workaround screenshot/pen-paper :D)
4. - [ ] ability to continue benchmark. I'm testing multiple models but since it takes some time and suddenly I need resources for something else i canceled it. (routing isn't displayed and I can start from zero). More or so same as issue 3.
5. - [ ] ability to directly see whats model outputting during specific test (tbf I don't understand exactly how tests work if its comparing 1 to 1 output or functionality but would be cool to see reasoning of models behind the scenes, not necessary).

Other stuff mentioned in open issues already
1. - [ ] share inference benchmarks based on hardware stats... issue 3 in QOL. could be nice solution to this. see #867

### Proposed solution

TBD

### Alternatives considered

using `--bench` cli flag, but still limited to some extent

### Feature area

TUI (new keybinding, view, or interaction)

### Would you be willing to contribute this?

No, just suggesting

### Additional context

I'm on version 1.1.10. This is so far what I was able to grasp. Overly beautiful application. If something else comes to my mind and this won't be closed I might add here regarding inference bench.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.