AlexsJones / AlexsJones/llmfit
Detect NPUs and variant core specs?
- Langage dominant
- Rust
- Étoiles
- 36.3k
- Forks
- 2.3k
- Merge moyen
- 2 j 18 h
- PR mergées (30 j)
- 85
Description
Running on a Rockchip RK3588 board, llmfit shows "GPU: none". But that chip has an onboard Arm Mali-G610 MP4 GPU as well as a 6 TOPS NPU which can accelerate model performance (though it does require preprocessing of models to NPU format). Also the CPU cores are divided into 4 each of two performance classes, just like Apple's M* chips.
Is it feasible for llmfit to incorporate this data in its model capacity and speed estimates?
For reference here is a ChatGPT answer on detection of these system features, FWIW:
https://chatgpt.com/s/t_69a316ce1b0481918746b2fd988fd584
Guide de contribution
Ouvrir le guide de contribution
Évaluation
Cette issue n'a pas encore été évaluée.