AlexsJones / AlexsJones/llmfit

Detect NPUs and variant core specs?

Ouverte
#122 3 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
Feasibility Feature Request Hardware Compatibility
Langage dominant
Rust
Étoiles
36.3k
Forks
2.3k
Merge moyen
2 j 18 h
PR mergées (30 j)
85

Description

Running on a Rockchip RK3588 board, llmfit shows "GPU: none". But that chip has an onboard Arm Mali-G610 MP4 GPU as well as a 6 TOPS NPU which can accelerate model performance (though it does require preprocessing of models to NPU format). Also the CPU cores are divided into 4 each of two performance classes, just like Apple's M* chips.

Is it feasible for llmfit to incorporate this data in its model capacity and speed estimates?

For reference here is a ChatGPT answer on detection of these system features, FWIW:

https://chatgpt.com/s/t_69a316ce1b0481918746b2fd988fd584

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.