Invalid tests prior to 0.9.3 due to NVIDIA CUDA ggml-cuda.dll performance

The speed difference between a locally-compiled, dynamically-linked ggml-cuda.dll (a la version 0.9.3) and the included ggml-cuda.dll (in version 0.9.2) makes older results invalid for users and your website.

The database should be purged of any old results that likely used the open-source version of ggml-cuda.dll, which was shipped in earlier versions of llamafile/localscore.

After I discovered that the latest Windows version (0.9.3) forces me to install both NVIDIA's CUDA toolkit and the Microsoft Visual Studio commandline, I decided to test the 0.9.2 version after my 0.9.3 results were uploaded to the server. I removed all PATH elements to the CUDA and MSVC software. This tricked llamafile to put its own ggml-cuda.dll in my user folder, which I confirmed by ensuring that compilation did not take place.

I did two tests. It's not pretty.

Using the localscore numbers on my screen in purple, I calculated percent change as (2nd - 1st)/1st, with the 2nd results being the slower version 0.9.2.

For one model, it was -56% and another was -54%.

That means the CUDA/MSVC version is about 2.1 times faster than the open source model. This is basically true for the token speed.

I measured a percent difference of 159% for "first time to token" for one model, meaning that the user waits 2.59 times longer than he would for a system-compiled dll.

I did not make the results public for my 2nd tests, just my 1st ones.

I would recommend collecting and publishing more key details during benchmarking: available system RAM, OS, and benchmark version.

I would also warn everyone to always use llamafile with the closed-source dll, which may have licensing issues for some edge use cases.

Start by reviewing the benchmark database and the result collection and publication flow for records produced before version 0.9.3. Done means removing results likely affected by the older ggml-cuda.dll and determining how system RAM, OS, and benchmark version can be collected and published.