Upgrading to Rust 1.94 from 1.93.1 causes severe runtime regression on 2-socket EPYC 9754 (Bergamo)

Upgrading from Rust 1.93.1 to Rust 1.94.0 causes a ~50% runtime performance regression for the chess engine Reckless on a 2×EPYC 9754 (Bergamo) machine when compiled with PGO. Both compiler versions ship the same LLVM (21.1.8), so the cause must be a rustc-level change.

This regression caused the TCEC Season 29 Superfinal - the premier computer chess championship - to be restarted after the engine was observed running at roughly half its expected speed.

Version it worked on

Rust 1.93.1 (rustc 1.93.1 (01f6ddf75 2026-02-11), LLVM 21.1.8)

Version with regression

rustc 1.94.0 (4a4ef493e 2026-03-02)
binary: rustc
commit-hash: 4a4ef493e3a1488c6e321570238084b38948f6db
commit-date: 2026-03-02
host: x86_64-unknown-linux-gnu
release: 1.94.0
LLVM version: 21.1.8

Key finding: identical LLVM versions

rustc 1.93.1  →  LLVM 21.1.8  →  WORKS (174M nps on TCEC machine)
rustc 1.94.0  →  LLVM 21.1.8  →  BROKEN (~77M nps on TCEC machine)

Since both versions ship the same LLVM, the regression must be in rustc-level changes between 1.93.1 and 1.94.0.

Runtime impact

DivP: https://github.com/codedeliveryservice/Reckless/commits/divp
Sufi: https://github.com/codedeliveryservice/Reckless/commits/sufi/
Run make for the binaries

Tested on the actual TCEC tournament machine (2×EPYC 9754, 512 threads), by TCEC admin Aloril, all built with PGO on the TCEC machine:

The third row is critical - even older source code becomes slower when compiled with 1.94.0 + PGO, confirming this is a compiler regression.

The runtime regression only occurs on the TCEC machine: 2×EPYC 9754 (Bergamo/Zen 4c, 2MB L3 per core). It does not appear on our other machines:

• 2×EPYC 9654 (Genoa/Zen 4, 4MB L3 per core)
• 1×EPYC 9754 (single socket Bergamo)

Note: both multi-socket machines use non-default BIOS NUMA configurations (32 nodes on the 2×9754, 24 nodes on the 2×9654). This was investigated and ruled out as a direct cause.

Workaround

Compiling with Rust 1.93.1 avoids the regression entirely.

Start with the Reckless DivP and SuFi commits linked in the issue and run make using Rust 1.93.1 and 1.94.0 with PGO on the described 2×EPYC 9754 system. Compare the results and investigate rustc changes between those releases; done means identifying the compiler change responsible for the regression or producing a focused reproduction.