Potential issue with dmypy file save status not being updated properly
Nessuno ha ancora preso questa issue.
- Lingua principale
- Python
- Stelle
- 20.6k
- Fork
- 3.3k
- Metriche di merge delle PR
- Metriche PR in attesa
Descrizione
Bug Report
To be completely honest I am hesitant to open this issue, as I am unable to reproduce it, but it was really strange so I'm going to anyways.
I ran dmypy suggest on line 108, in example below that's in the function pair_encode_stream on the line for char in text: after previously having a successful typecheck with the following command:
dmypy --status-file="/home/<my username>/.idlerc/mypy/dmypy.json" run --timeout=60 --log-file="/home/<my username>/.idlerc/mypy/log.txt" --export-types "/home/<my username>/Desktop/Python/Tools/byte_pair_encoding.py" -- --warn-unused-ignores --show-absolute-path --disallow-untyped-defs --show-traceback --warn-unreachable --show-error-codes --cache-fine-grained --no-implicit-reexport --warn-redundant-casts --show-error-end --cache-dir="/home/<my username>/.idlerc/mypy" --soft-error-limit=-1 --disallow-untyped-calls --strict --no-error-summary --hide-error-context --no-color-output --show-column-numbers
but had since continued adding lines to the program. I hadn't realized as of yet that line 130 would result in a syntax error.
Apon running dmypy suggest, I got this response from it:
{
"func_name": "pair_decode",
"line": 105,
"path": "/home/<my username>/Desktop/Python/Tools/byte_pair_encoding.py",
"samples": 0,
"signature": {
"arg_types": [
"bytes",
"typing.Dict[int, bytes]"
],
"return_type": "None"
}
}
claiming that the function definition start was on line 105 instead of line 103.
My dmypy extension raised an error about being unable to parse the function definition while attempting to add dmypy's suggestion, which I thought was "aw man looks like I have a bug again" and tried running dmypy suggest on line 104, and
this time it returned something similar to this (I did not record original response at the time):
{
"func_name": "pair_encode",
"line": 70,
"path": "/home/<my username>/Desktop/Python/Tools/byte_pair_encoding.py",
"samples": 0,
"signature": {
"arg_types": [
"str"
],
"return_type": "Tuple[bytes, typing.Dict[int, bytes]]"
}
}
notibly being the proper definition start for pair_encode_stream instead of pair_decode
To Reproduce
Exact state of the file when issue was happening
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""TITLE - DESCRIPTION"""
# Programmed by CoolCat467
from __future__ import annotations
# TITLE - DESCRIPTION
# Copyright (C) 2024 CoolCat467
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
__title__ = "Byte pair encoding AKA digram coding"
__author__ = "CoolCat467"
__version__ = "0.0.0"
__license__ = "GNU General Public License Version 3"
from collections import Counter
from collections.abc import Generator, Sequence, Buffer
from typing import TypeVar, TypeGuard
import string
T = TypeVar("T")
def get_groups(data: Sequence[T], n: int = 2) -> Generator[Sequence[T], None, None]:
if len(data) < n:
raise ValueError(f"Cannot be length less than {n}")
for i in range(len(data) - n):
yield data[i:i+n]
yield data[-n:]
def get_pairs(string: str) -> Generator[str, None, None]:
if len(string) < 2:
raise ValueError("Cannot be length less than 2")
buffer = [string[0], string[1]]
for char in string[2:]:
yield "".join(buffer)
buffer[0] = buffer[1]
buffer[1] = char
yield "".join(buffer)
def prove_is_bytes(value: Sequence[int]) -> TypeGuard[bytes]:
return isinstance(value, bytes)
def prove_bytes_counter(value: Counter[Sequence[int]]) -> TypeGuard[Counter[bytes]]:
for key in value:
if not prove_is_bytes(key):
return False
return True
def pair_encode(text: str) -> tuple[bytes, dict[int, bytes]]:
transformed = text.encode("utf-8")
usable = set(range(256)) - set(transformed)
encode_map: dict[int, bytes] = {}
while True:
pairs = Counter(get_groups(transformed, 2))
if not prove_bytes_counter(pairs):
break
most_common, times = pairs.most_common(1)[0]
if times < 2:
break
replace = usable.pop()
encode_map[replace] = most_common
transformed = transformed.replace(most_common, replace.to_bytes())
return transformed, encode_map
def pair_encode_stream(text: str) -> Generator[tuple[int, bytes] | bytes, None, None]:
transformed = text.encode("utf-8")
usable = set(range(256)) - set(transformed)
while True:
pairs = Counter(get_groups(transformed, 2))
if not prove_bytes_counter(pairs):
break
most_common, times = pairs.most_common(1)[0]
if times < 2 or not usable:
break
replace = usable.pop()
yield replace, most_common
transformed = transformed.replace(most_common, replace.to_bytes())
yield transformed
def pair_decode(transformed: bytes, decode_map: dict[int, bytes]):
text = transformed
matched = True
while matched:
matched = False
for char in text:
if char not in decode_map:
continue
matched = True
text = text.replace(char.to_bytes(), decode_map[char])
return text.decode("utf-8")
# types: misc error: The return type of a generator function should be "Generator" or one of its supertypes
def test() -> None:
yield 3
yield 4
return 5
def run() -> None:
"Run program"
transformed, encode_map = pair_encode("aaabdaaabac")
print((transformed, encode_map))
# types: func-returns-value error: "pair_decode" does not return a value (it only ever returns None)
print(f'{pair_decode(transformed, encode_map) = }')
# types: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
for value in pair_encode_stream("waffles are all the rage in mississippi"):
# types: syntax error: expected an indented block after 'for' statement on line 129
# types: syntax error: expected an indented block after 'for' statement on line 130
if __name__ == "__main__":
print(f"{__title__} v{__version__}\nProgrammed by {__author__}.\n")
run()
And I have no idea if this is relevant, but then apon realizing this is a dmypy bug and copying the program to a different folder and attempting to reproduce this,
the dmypy daemon crashed and apon restarting it and checking the file it told me about the syntax issue.
This is the most recent traceback in log.txt, specified in run arguments.
To be honest not sure if this is related though.
Traceback (most recent call last):
File "mypy/dmypy_server.py", line 236, in serve
File "mypy/dmypy_server.py", line 285, in run_command
File "mypy/dmypy_server.py", line 353, in cmd_run
File "mypy/dmypy_server.py", line 432, in check
File "mypy/dmypy_server.py", line 698, in fine_grained_increment_follow_imports
File "mypy/server/update.py", line 267, in update
File "mypy/server/update.py", line 369, in update_one
File "mypy/server/update.py", line 452, in update_module
File "mypy/server/update.py", line 881, in propagate_changes_using_dependencies
File "mypy/server/update.py", line 1025, in reprocess_nodes
File "mypy/checker.py", line 535, in check_second_pass
File "mypy/checker.py", line 540, in check_partial
File "mypy/checker.py", line 554, in check_top_level
File "mypy/nodes.py", line 1351, in accept
File "mypy/checker.py", line 2934, in visit_assignment_stmt
File "mypy/checker.py", line 2974, in check_type_alias_rvalue
File "mypy/checkexpr.py", line 5832, in accept
File "mypy/errors.py", line 1269, in report_internal_error
File "mypy/checkexpr.py", line 5830, in accept
File "mypy/nodes.py", line 2112, in accept
File "mypy/checkexpr.py", line 3379, in visit_op_expr
File "mypy/checkexpr.py", line 5832, in accept
File "mypy/errors.py", line 1269, in report_internal_error
File "mypy/checkexpr.py", line 5830, in accept
File "mypy/nodes.py", line 2714, in accept
File "mypy/checkexpr.py", line 4730, in visit_type_alias_expr
File "mypy/checkexpr.py", line 4794, in alias_type_in_runtime_context
File "mypy/checker.py", line 6939, in named_generic_type
File "mypy/checker.py", line 6946, in lookup_typeinfo
File "mypy/checker.py", line 7024, in lookup_qualified
KeyError: 'types'
Your Environment
- Mypy version used:
dmypy 1.11.0 - Mypy command-line flags: See above
- Mypy configuration options from
mypy.ini(and other config files): No other configuration files active - Python version used:
Python 3.12.0 (main, Apr 28 2024, 22:42:26) [GCC 13.2.0] on linux
Guida per i contributori
Apri la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Direzione di ricerca
Inizia riproducendo il comportamento di dmypy run e suggest con il file Python fornito e i flag del comando, quindi esamina mypy/dmypy_server.py e il percorso di aggiornamento in mypy/server/update.py. Il traceback indica anche checker.py e checkexpr.py. Il lavoro è completato quando lo stato obsoleto o il suggerimento errato segnalato è stato riprodotto e la relativa gestione è coperta senza il crash del daemon descritto qui.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- python
- Ambito
- devtools, tooling
- Tipo di issue
- Bug
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Stato di attività
- Ferma
- Chiarezza
- Da chiarire
- Idoneità per principianti
- 25/100