AMD GPU scores probably not representative of actual performance

I just ran localscore, and submitted a score here: https://www.localscore.ai/result/492

Despite being long context results, they still seemed a lot lower than expected, particularly the prompt processing.

For my 7900 XTX, using a recent llama.cpp build: https://github.com/ggml-org/llama.cpp/releases/tag/b5062
(llama-b5062-bin-win-hip-x64-gfx1100.zip)

llama-bench on the same model gets a pp1024 of 2786 tok/s vs the LocalScore result of 1068 tok/s

For tg1024, doing some math, for the pp difference, output is about 83.1 tok/s vs LocalScore's 50.9 tok/s result.

This tracks as the tg1024 result on llama-bench is ~91.8 tok/s and the pp64+tg1024 that LocalScore reports is... 68.4 tok/s.

Not sure where llamafile actually forks off, but just an FYI that it seems to be a lot slower than llama.cpp proper.

llama-b5062-bin-win-hip-x64-gfx1100>llama-bench -m ..\..\Downloads\Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf -ngl 99 -p 1024 -n 1024 -pg 1024,1024 -o json
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
  Device 0: AMD Radeon RX 7900 XTX, gfx1100 (0x1100), VMM: no, Wave Size: 32
[
  {
    "build_commit": "d0d5b223",
    "build_number": 5062,
    "cpu_info": "AMD Ryzen 7 5800X3D 8-Core Processor           ",
    "gpu_info": "AMD Radeon RX 7900 XTX",
    "backends": "ROCm,RPC",
    "model_filename": "..\\..\\Downloads\\Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf",
    "model_type": "llama 8B Q4_K - Medium",
    "model_size": 4912898304,
    "model_n_params": 8030261312,
    "n_batch": 2048,
    "n_ubatch": 512,
    "n_threads": 8,
    "cpu_mask": "0x0",
    "cpu_strict": false,
    "poll": 50,
    "type_k": "f16",
    "type_v": "f16",
    "n_gpu_layers": 99,
    "split_mode": "layer",
    "main_gpu": 0,
    "no_kv_offload": false,
    "flash_attn": false,
    "tensor_split": "0.00",
    "use_mmap": true,
    "embeddings": false,
    "n_prompt": 1024,
    "n_gen": 0,
    "test_time": "2025-04-07T12:32:24Z",
    "avg_ns": 367549380,
    "stddev_ns": 1377953,
    "avg_ts": 2786.051618,
    "stddev_ts": 10.472910,
    "samples_ns": [ 365338900, 368401900, 368955400, 367480800, 367569900 ],
    "samples_ts": [ 2802.88, 2779.57, 2775.4, 2786.54, 2785.86 ]
  },
  {
    "build_commit": "d0d5b223",
    "build_number": 5062,
    "cpu_info": "AMD Ryzen 7 5800X3D 8-Core Processor           ",
    "gpu_info": "AMD Radeon RX 7900 XTX",
    "backends": "ROCm,RPC",
    "model_filename": "..\\..\\Downloads\\Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf",
    "model_type": "llama 8B Q4_K - Medium",
    "model_size": 4912898304,
    "model_n_params": 8030261312,
    "n_batch": 2048,
    "n_ubatch": 512,
    "n_threads": 8,
    "cpu_mask": "0x0",
    "cpu_strict": false,
    "poll": 50,
    "type_k": "f16",
    "type_v": "f16",
    "n_gpu_layers": 99,
    "split_mode": "layer",
    "main_gpu": 0,
    "no_kv_offload": false,
    "flash_attn": false,
    "tensor_split": "0.00",
    "use_mmap": true,
    "embeddings": false,
    "n_prompt": 0,
    "n_gen": 1024,
    "test_time": "2025-04-07T12:32:26Z",
    "avg_ns": 11160464920,
    "stddev_ns": 24885413,
    "avg_ts": 91.752815,
    "stddev_ts": 0.204563,
    "samples_ns": [ 11130246600, 11142845900, 11161517600, 11174967700, 11192746800 ],
    "samples_ts": [ 92.0016, 91.8975, 91.7438, 91.6334, 91.4878 ]
  },
  {
    "build_commit": "d0d5b223",
    "build_number": 5062,
    "cpu_info": "AMD Ryzen 7 5800X3D 8-Core Processor           ",
    "gpu_info": "AMD Radeon RX 7900 XTX",
    "backends": "ROCm,RPC",
    "model_filename": "..\\..\\Downloads\\Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf",
    "model_type": "llama 8B Q4_K - Medium",
    "model_size": 4912898304,
    "model_n_params": 8030261312,
    "n_batch": 2048,
    "n_ubatch": 512,
    "n_threads": 8,
    "cpu_mask": "0x0",
    "cpu_strict": false,
    "poll": 50,
    "type_k": "f16",
    "type_v": "f16",
    "n_gpu_layers": 99,
    "split_mode": "layer",
    "main_gpu": 0,
    "no_kv_offload": false,
    "flash_attn": false,
    "tensor_split": "0.00",
    "use_mmap": true,
    "embeddings": false,
    "n_prompt": 1024,
    "n_gen": 1024,
    "test_time": "2025-04-07T12:33:22Z",
    "avg_ns": 12691306920,
    "stddev_ns": 27204594,
    "avg_ts": 161.370893,
    "stddev_ts": 0.345830,
    "samples_ns": [ 12663685400, 12666764500, 12718059400, 12720693700, 12687331600 ],
    "samples_ts": [ 161.722, 161.683, 161.031, 160.998, 161.421 ]
  }
]

Start with the linked LocalScore result and the supplied llama-bench command and output, then trace the benchmark entry point that produces the pp and tg1024 measurements. Reproduce the AMD Radeon RX 7900 XTX comparison and determine whether the discrepancy is in the benchmark configuration or execution; done when the cause is documented or the scores are representative.