python / python/cpython

Tools/ftscalingbench picks too few CPUs on processors with unevenly clocked performance cores

Aberta
#155,090 0 comentários 0 reações 0 responsáveis Ver no GitHub

Ninguém assumiu esta issue ainda.

topic-free-threading type-bug
Linguagem predominante
Python
Estrelas
77.2k
Forks
36k
Métricas de merge de PRs
Métricas de PR pendentes

Descrição

Tools/ftscalingbench chooses which CPUs to run its worker threads on by reading lscpu -p=cpu,node,core,MAXMHZ and keeping one CPU per physical core, skipping efficiency cores. The filter for that last part is an exact comparison against the highest clock on the machine:

max_mhz_all = max(row[3] for row in table)
for cpu, node, core, maxmhz in table:
    if node == 0 and core not in cores and maxmhz == max_mhz_all:

That works when every performance core shares one clock ceiling. It does not hold on parts with Intel Turbo Boost Max 3.0, which bins a couple of favoured cores above their siblings. Only those few then pass the filter, and the rest of the performance cores are dropped along with the efficiency cores.

On an i7-14650HX (8 performance cores, two of them rated 5200 MHz and the other six 5000 MHz, plus 8 efficiency cores at 3700 MHz) the selection returns [8, 10] and the benchmark prints:

Running benchmarks with 2 threads

It then reports scaling figures for 2 threads on a machine with 8 performance cores, without any indication that it picked a smaller thread count than the hardware offers. The numbers are not wrong for what was measured; they just are not measuring what the person running it expects.

This needs no particular hardware to see. Run the following from a checkout, which feeds the selection function that machine's topology:

import sys
from unittest import mock
sys.path.insert(0, "Tools/ftscalingbench")
import ftscalingbench

LSCPU = "# cpu,node,core,MAXMHZ\n" + "".join(
    f"{c},0,{c//2},{'5200.0000' if c//2 in (4, 5) else '5000.0000'}\n"
    for c in range(16)
) + "".join(f"{16+i},0,{8+i},3700.0000\n" for i in range(8))

with (mock.patch("subprocess.check_output", return_value=LSCPU),
      mock.patch.object(sys, "platform", "linux")):
    print(ftscalingbench.determine_num_threads_and_affinity())

Expected [0, 2, 4, 6, 8, 10, 12, 14], one CPU per performance core. Actual [8, 10].

Any CPU whose performance cores are not all binned to the same clock runs into this. I have only confirmed it on the Intel part above; whether other heterogeneous designs report distinct MAXMHZ values within their fast cluster I have not checked, though where they do, the same filter would keep only the fastest of them.

Splitting performance and efficiency cores at the midpoint between the highest and lowest reported clock handles the uneven binning, since efficiency cores sit far below every performance core on the parts this code is trying to account for. Machines that report one clock for every core, or no clock at all, need to keep accepting everything, which is the case in many virtual machines where MAXMHZ comes back empty.

This only affects people running the benchmark by hand on Linux; it is not part of pyperformance or any automated suite. It does mean that free-threading scaling measurements taken on a recent Intel laptop or desktop may have been collected with fewer threads than intended.

I have a patch and tests for this and will open a PR.

Linked PRs
  • gh-155105

Guia de contribuição

Abrir o guia de contribuição

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Direção de pesquisa

Comece em Tools/ftscalingbench e determine_num_threads_and_affinity(), depois execute a reprodução mock fornecida de lscpu. Está concluído quando a topologia reproduzida seleciona [0, 2, 4, 6, 8, 10, 12, 14], enquanto dados de clock uniformes, ausentes ou de máquina virtual continuam sendo aceitos; revise o patch e os testes associados a PR gh-155105.

Escrita pelo modelo de indexação a partir do texto da issue.

Avaliação

Stack de tecnologia
python
Domínio
performance, tooling
Tipo de issue
Bug
Dificuldade
3/5
Tempo estimado
1-2 dias
Status de atividade
Estagnada
Clareza
Claramente especificada
Facilidade para iniciantes
15/100

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.