Replace module level mutable containers with immutable containers
Nadie ha tomado este issue todavía.
- Lenguaje dominante
- Python
- Estrellas
- 77.2k
- Forks
- 35.9k
- Métricas de merge de PR
- Métricas de PR pendientes
Descripción
There are many module level containers with constants. By replacing the mutable containers with immutable variants (e.g. replace a set with a frozenset or a list with a tuple) we improve performance (especially in the free-threaded build, see for example https://github.com/python/cpython/pull/138429) and avoid accidental modification of these containers.
The number of module level lists, dicts and sets on current main is:
number of mutable module level containers by type:
<class 'dict'>: 266
<class 'list'>: 150
<class 'set'>: 63
<class 'collections.defaultdict'>: 2
<class '_strptime.TimeRE'>: 1
<class 'email._encoded_words._QByteMap'>: 1
Script to list all the module level mutable containers
import sys
import importlib
import pkgutil
from collections import Counter
blacklist = ('idlelib.idle')
excluded_submodule_names = ('__main__')
search_submodules = 2
mutable_containers = (list, dict, set)
def list_container_types(module, mcc):
print_module = False
for a in dir(module):
if a in ('__all__', '__path__', '__builtins__', '__annotations__', '__conditional_annotations__'):
# why is __all__ a list and not a tuple?
continue
# if not a.startswith('_'):
# continue
attr = getattr(module, a)
tp = type(attr)
if issubclass(tp, mutable_containers):
if not print_module:
print(f'{module}:')
print_module = True
print(f' {a}: {tp}')
mcc.update([tp])
def search_modules(module_names, search_submodules: int, mcc):
for name in module_names:
if name in blacklist:
continue
try:
module = importlib.import_module(name)
except:
print(f'{name}: error on import')
module = None
list_container_types(module, mcc)
if search_submodules:
try:
sub_names = list(z.name for z in pkgutil.iter_modules(module.__path__))
except Exception as ex:
sub_names = []
mm = [name + '.' + sub_name for sub_name in sub_names if sub_name not in excluded_submodule_names]
search_modules(mm, search_submodules - 1, mcc)
mcc = Counter()
module_names = sorted(list(sys.builtin_module_names)) + sorted(list(sys.stdlib_module_names))
search_modules(module_names, search_submodules=2, mcc = mcc)
print()
print('number of module level containers by type:')
for key, value in mcc.items():
print(f'{key}: {value}')
Not all the mutable containers can be replaced by immutable containers. Some of them need to be mutable (e.g. copyreg.dispatch_table). And some of them are part of the public API and we might not want to change the type only for performance reasons.
Example candidates: _pydatetime._DAYNAMES (would improve performance of date.cdate), token.EXACT_TOKEN_TYPES, xml.etree.ElementTree.HTML_EMPTY
Linked PRs
- gh-139004
Guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Línea de trabajo
El issue proporciona un script para escanear módulos y nombra contenedores candidatos, incluidos _pydatetime._DAYNAMES, token.EXACT_TOKEN_TYPES y xml.etree.ElementTree.HTML_EMPTY. Empieza revisando el PR vinculado gh-139004 y comprobando la mutabilidad y el uso en la API pública de cada candidato. El trabajo estará terminado cuando los contenedores adecuados a nivel de módulo se hayan convertido de forma segura sin cambiar el comportamiento requerido.
Escrito por el modelo de indexación a partir del texto del issue.
Evaluación
- Stack tecnológico
- python
- Área
- compilers
- Tipo de issue
- Refactorización
- Dificultad
- 5/5
- Tiempo estimado
- Más de una semana
- Estado de actividad
- Estancado
- Claridad
- Necesita aclaración
- Aptitud para principiantes
- 25/100