abetlen / abetlen/llama-cpp-python

Load time in the log is much different from real load model time?

Open
#1,601 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
10.6k
Forks
1.4k
PR merge metrics
PR metrics pending

Description

I see that load time in the log is much smaller than real load time.
I calculate real oad model time by setting time before and after calling Llama-cpp class. From here, I saw thật, real load model time is much larger than the load time in the log.

Could you help me, what is wrong here? Thank you.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.