Pure-Python `date.fromisoformat` silently mis-parses malformed basic-format dates
Nadie ha tomado este issue todavía.
- Lenguaje dominante
- Python
- Estrellas
- 77.2k
- Forks
- 35.9k
- Métricas de merge de PR
- Métricas de PR pendientes
Descripción
Bug description
_pydatetime.date.fromisoformat (the pure-Python reference used when the C
accelerator is unavailable, and via _pydatetime directly) returns a
wrong-but-plausible date for several strings that are not valid ISO-8601
dates. The C accelerator raises ValueError for every one of them. Because the
result is a silently incorrect date rather than an error, malformed input
becomes valid-looking data with no signal that anything went wrong.
There are two surface forms of the same underlying defect in
_parse_isoformat_date, which slices fixed-width substrings and calls int()
on them without checking that each slice is exactly N ASCII digits:
(1) int() tolerates a leading + / - / space in a basic-format field.
>>> import _datetime, _pydatetime
>>> _pydatetime.date.fromisoformat('2020+12')
datetime.date(2020, 1, 2)
>>> _datetime.date.fromisoformat('2020+12')
Traceback (most recent call last):
...
ValueError: Invalid isoformat string: '2020+12'
>>> _pydatetime.date.fromisoformat('+020-06-15')
datetime.date(20, 6, 15)
>>> _pydatetime.date.fromisoformat('2020-W 5')
datetime.date(2020, 1, 27)
>>> _pydatetime.date.fromisoformat('202012+9')
datetime.date(2020, 12, 9)
>>> _pydatetime.date.fromisoformat('2020 12')
datetime.date(2020, 1, 2)
Here int('+1') == 1, int(' 1') == 1 and int('+9') == 9, so the month /
day / week fields parse a sign or space that is not part of any ISO-8601 date.
(2) The length gate admits 7-character strings, and a fixed-width slice then
reads a 1-character tail.
>>> _pydatetime.date.fromisoformat('2020061')
datetime.date(2020, 6, 1)
>>> _datetime.date.fromisoformat('2020061')
Traceback (most recent call last):
...
ValueError: Invalid isoformat string: '2020061'
>>> _pydatetime.date.fromisoformat('2020123')
datetime.date(2020, 12, 3)
>>> _pydatetime.date.fromisoformat('2020-W2')
datetime.date(2020, 1, 6)
>>> _pydatetime.date.fromisoformat('9999121')
datetime.date(9999, 12, 1)
'2020061' is 7 chars; the gate len(date_string) in (7, 8, 10) lets it
through, the month slice reads '06' and the day slice dtstr[6:8] reads the
1-character tail '1', giving date(2020, 6, 1). '2020-W2' reads a 1-digit
week int('2'). The C parse_digits(p, ..., 2) requires exactly two digits, so
C rejects all of these.
datetime.fromisoformat inherits the same defect via the date branch, e.g.
_pydatetime.datetime.fromisoformat('2020061') returns
datetime.datetime(2020, 6, 1, 0, 0) while the C path raises.
C vs pure-Python
| input | C _datetime |
pure-Python _pydatetime |
|---|---|---|
date.fromisoformat('2020+12') |
ValueError |
date(2020, 1, 2) |
date.fromisoformat('+020-06-15') |
ValueError |
date(20, 6, 15) |
date.fromisoformat('2020-W 5') |
ValueError |
date(2020, 1, 27) |
date.fromisoformat('202012+9') |
ValueError |
date(2020, 12, 9) |
date.fromisoformat('2020061') |
ValueError |
date(2020, 6, 1) |
date.fromisoformat('2020123') |
ValueError |
date(2020, 12, 3) |
date.fromisoformat('2020-W2') |
ValueError |
date(2020, 1, 6) |
date.fromisoformat('9999121') |
ValueError |
date(9999, 12, 1) |
Root cause
Lib/_pydatetime.py, _parse_isoformat_date (the function's own comment notes
it "assumes an ASCII-only string of lengths 7, 8 or 10"). On current main the
function body is:
def _parse_isoformat_date(dtstr):
# It is assumed that this is an ASCII-only string of lengths 7, 8 or 10,
# see the comment on Modules/_datetimemodule.c:_find_isoformat_datetime_separator
if len(dtstr) not in (7, 8, 10): # line 361
raise ValueError("Invalid isoformat string")
year = int(dtstr[0:4]) # line 363
...
weekno = int(dtstr[pos:pos + 2]) # line 370 (week field)
...
dayno = int(dtstr[pos:pos + 1]) # line 380 (week day field)
...
month = int(dtstr[pos:pos + 2]) # line 384 (month field)
...
day = int(dtstr[pos:pos + 2]) # line 390 (day field)
The if len(dtstr) not in (7, 8, 10) gate at line 361 only bounds the total
length; date.fromisoformat (the caller, lines 1059-1060) applies the same
length gate before calling in. Neither gate checks the content of the
fixed-width fields. Each field is read with int(dtstr[pos:pos+N]). int()
accepts a leading +/-/whitespace and a short string, so:
- a
+/-/space that lands in a month/day/week field is silently consumed
(form 1), and - on a 7-char string the day/week slice runs off the end and
int()happily
parses the 1-character remainder (form 2).
Wrong side: pure-Python, which over-accepts. ISO-8601 calendar dates contain
no sign or space inside the date, and have no 1-digit month-day or 1-digit
week; date.fromisoformat's docstring promises a string "in the format emitted
by date.isoformat()". The C accelerator's parse_digits rejects any non-digit
byte and requires the exact field width, then verifies the whole string was
consumed, so C is correct here.
Suggested fix
Validate each slice in _parse_isoformat_date before converting: require that
the year/month/day/week/weekday slice is exactly N ASCII digits (mirroring the
C parse_digits). The module already has an _is_ascii_digit helper used by
the fraction path (_parse_hh_mm_ss_ff does
all(map(_is_ascii_digit, tstr[pos:]))), so reusing it keeps the check
consistent, e.g. raise ValueError unless
len(s) == N and all(map(_is_ascii_digit, s)) before calling int(s). That
makes the malformed basic-format strings above raise ValueError on the
pure-Python path exactly as the C accelerator does, and closes the 7-char
short-slice hole at the same time. (The length gate (7, 8, 10) can stay; the
per-field width check is what rejects '2020061', since its day slice is then a
1-char string.)
Environment
- Reproduced on current
main,3.16.0a0. - On current
main, the length gate in_parse_isoformat_dateis already an
if len(dtstr) not in (7, 8, 10): raise ValueError(gh-152060 / PR #152061,
merged, replaced the earlierassert len(dtstr) in (7, 8, 10)so that a bad
length raisesValueErrorinstead ofAssertionError). That change only
touched the length-gate exception type; it did not touch the
int(dtstr[...])slices, so both mis-parse forms above still reproduce on
main. (A pre-PR-#152061 checkout still carries theassert; the slice
defect is the same either way.) - The same slice-based
_parse_isoformat_dateexists in the 3.14 / 3.15
branches (the pure-Python module is_pydatetimein all of them), so those
branches are affected wherever the pure-Python path is exercised (the C
accelerator masks it when present).
Relation to existing issues
This is distinct from the known nearby issues:
- gh-107779 ("incorrectly accepts and parses strings without date-time
separator", open; PR #107791) is about_find_isoformat_datetime_separator
returning an index that is not a real separator indatetime.fromisoformat.
That is a different function and mechanism; it does not address the
int()-slice leniency in_parse_isoformat_date. Verified on this build that
'2024-01-17T15:21:00-0800'(the basic/extended-mixing class) is accepted by
both implementations, i.e. not a C-vs-pure-Python divergence. - gh-152060 / PR #152061 (merged) was the exception-type fix: a
wrong-lengthdtstrused to raiseAssertionError(from the old
assert len(...)); PR #152061 turned that into aValueError. That is a
disjoint defect: it is about the length gate's exception type, whereas this
issue is about strings of a valid length (7/8/10) whose fixed-width fields
are mis-sliced into a silently wrong value. The per-field slices PR #152061
left untouched are exactly the ones at fault here.
Found with a differential C-vs-pure-Python fromisoformat testing harness
(AI-assisted, each case hand-verified).
Linked PRs
- gh-152205
- gh-156359
- gh-156367
- gh-156368
Guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Línea de trabajo
Comienza en Lib/_pydatetime.py, en _parse_isoformat_date, y compara su análisis de ancho fijo con el comportamiento de C parse_digits descrito en el issue. Verifica las entradas malformadas de formato básico indicadas y, después, añade validación del ancho de los campos y de los dígitos ASCII para que las rutas de date y datetime de Python puro generen ValueError como _datetime.
Escrito por el modelo de indexación a partir del texto del issue.
Evaluación
- Stack tecnológico
- python
- Área
- backend
- Tipo de issue
- Error
- Dificultad
- 3/5
- Tiempo estimado
- 1-2 días
- Estado de actividad
- Estancado
- Claridad
- Bien especificado
- Aptitud para principiantes
- 25/100