compute-eval: correctness-focused benchmark, not performance-oriented?

Hi, thanks for releasing compute-eval, it’s a very useful tool.

After reviewing the codebase, it seems that the benchmark primarily measures end-to-end time from compilation to execution, without reporting the actual CUDA kernel runtime. Moreover, the evaluation logic centers on pass@k correctness.

This makes compute-eval appear to be mainly a correctness benchmark: assessing whether models can generate valid CUDA kernels, rather than a performance-oriented benchmark like KernelBench or robust-kbench, which evaluate kernel execution speed and optimization quality.

Could you clarify if this correctness-first design is intentional? Are there any plans to incorporate performance metrics in future releases, so that it can also benchmark kernel optimization capabilities?

Start by reading compute_eval/execution.py around the end-to-end timing and compute_eval/evaluation.py around pass@k, then compare the benchmark scope described in the issue. Done would require a maintainer decision on whether performance metrics belong in the project and a defined, agreed scope for any follow-up work.