python / python/mypy

Daemon performance seems slow

Abierto
#16,745 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Nadie ha tomado este issue todavía.

performance topic-daemon
Lenguaje dominante
Python
Estrellas
20.6k
Forks
3.3k
Métricas de merge de PR
Métricas de PR pendientes

Descripción

I've written a simple little timing script:

import os
from timing import TicToc # our own timing context
with TicToc('stop daemon'):
    os.system('dmypy stop')
for _ in range(10):
    with TicToc('Run on scratch'):
        os.system('dmypy run scratch.py > boo.txt')
    with TicToc('Modify scratch'):
        with open('scratch.py', 'a') as f:
            f.write('\n\nx: int = 5\n')

scratch.py is a file in a repo with ~100k lines of code. It's a work in progress as far as typing goes; boo.txt ends with Found 2587 errors in 143 files (checked 1 source file)

The intention of the script is to simulate me working on code, e.g. scratch.py, and having type checking run frequently (e.g. every time I save the file).

With default mypy (1.8.0) settings, I get this:

Daemon stopped
[stop daemon] 0.161 seconds
[Run on scratch] 33.603 seconds
[Modify scratch] 0.002 seconds
[Run on scratch] 7.247 seconds
[Modify scratch] 0.002 seconds
[Run on scratch] 3.549 seconds
[Modify scratch] 0.001 seconds
[Run on scratch] 7.029 seconds
[Modify scratch] 0.002 seconds
[Run on scratch] 3.721 seconds
[Modify scratch] 0.001 seconds
[Run on scratch] 6.860 seconds
[Modify scratch] 0.002 seconds
[Run on scratch] 3.705 seconds
[Modify scratch] 0.001 seconds
[Run on scratch] 8.777 seconds
[Modify scratch] 0.002 seconds
[Run on scratch] 3.623 seconds
[Modify scratch] 0.001 seconds
[Run on scratch] 6.872 seconds
[Modify scratch] 0.002 seconds

In other words, the time it takes to check this one file (once the daemon has started) oscillates between ~4 seconds and ~7 seconds, just by adding a single line to scratch.py each time.

Without modifying scratch.py between runs, I get this:

Daemon stopped
[stop daemon] 0.161 seconds
[Run on scratch] 33.603 seconds
[Run on scratch] 4.002 seconds
[Run on scratch] 0.369 seconds
[Run on scratch] 0.364 seconds
[Run on scratch] 0.364 seconds
[Run on scratch] 0.371 seconds
[Run on scratch] 0.364 seconds
[Run on scratch] 0.366 seconds
[Run on scratch] 0.362 seconds
[Run on scratch] 0.362 seconds
  1. Is this oscillation in run time (seen on the first test) expected?
  2. Is it expected to see a second run be much slower than subsequent runs on the same file if nothing is modified?
  3. Is there anything I can do to speed this up, other than setting follow-imports to skip?

Guía de contribución

Abrir la guía de contribución

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Línea de trabajo

Comienza con el script de medición de tiempos proporcionado y compara ejecuciones repetidas de dmypy run scratch.py con y sin añadir contenido a scratch.py. Investiga los tiempos del daemon en la primera ejecución, en ejecuciones sin cambios y con el archivo modificado; se considera terminado cuando se explique la oscilación y se identifique alguna forma compatible de reducir el tiempo de comprobación más allá de follow-imports=skip.

Escrito por el modelo de indexación a partir del texto del issue.

Evaluación

Stack tecnológico
python
Área
devtools, performance
Tipo de issue
Error
Dificultad
4/5
Tiempo estimado
3-5 días
Estado de actividad
Estancado
Claridad
Necesita aclaración
Aptitud para principiantes
25/100

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.