Translate unsupported fsspec modification times into a typed DuckDB error
- Lenguaje dominante
- Python
- Estrellas
- 186
- Forks
- 113
- Merge medio
- 13 h 29 min
- PR fusionados (30 d)
- 17
Descripción
## What happens?
When a filesystem registered through `Connection.register_filesystem()` cannot provide a modification time, querying the `last_modified` column of `read_blob` surfaces the raw Python exception instead of returning `NULL`.
One concrete case is `gcsfs`: `GCSFileSystem.modified()` indexes `info(path)["mtime"]`, while some GCS object metadata (and synthesized directory entries) has no `mtime`. The resulting `KeyError: 'mtime'` currently reaches a DuckDB query as an untyped error and can abort callers such as DuckLake `CHECKPOINT`.
This is the Python/fsspec half of:
- duckdb/ducklake#1042
- duckdb/ducklake#1336
## Why the error loses its type
`PythonFilesystem::GetLastModifiedTime()` currently calls `filesystem.modified()` directly. A Python exception crosses the nanobind boundary as `std::exception`; when DuckDB later constructs `ErrorData(ex)`, it has `ExceptionType::INVALID`, so extensions cannot distinguish unsupported modification-time metadata from unrelated I/O failures without inspecting the message.
DuckDB core already has the desired downstream behavior: `DirectFileReader` converts `ExceptionType::NOT_IMPLEMENTED` failures for optional file metadata columns to `NULL`. An `older_than` predicate then naturally retains only files whose age is unknown.
## Suggested direction
Translate the Python filesystem capability error at the adapter boundary in `PythonFilesystem::GetLastModifiedTime()`:
1. Catch `nb::python_error`.
2. Map Python `NotImplementedError` to DuckDB `NotImplementedException`.
3. Consider a narrowly scoped compatibility mapping for the current gcsfs `KeyError('mtime')`, or coordinate with gcsfs so missing `mtime` is reported as `NotImplementedError`.
4. Re-throw every unrelated Python exception unchanged.
Please add a regression test with a small custom fsspec filesystem whose `modified()` is unsupported, asserting that `read_blob(...).last_modified` is `NULL` while other filesystem errors still surface.
This keeps provider-specific exception classification in the Python filesystem adapter instead of requiring downstream extensions to accumulate message-matching helpers.
Guía de contribución
Línea de trabajo
Comienza en PythonFilesystem::GetLastModifiedTime() y sigue cómo DirectFileReader gestiona los errores de metadatos opcionales. Añade una prueba de regresión usando un sistema de archivos fsspec personalizado con un comportamiento modified() no compatible y ejecútala mediante read_blob(...).last_modified. El trabajo estará terminado cuando las horas de modificación no compatibles devuelvan NULL, mientras que los errores del sistema de archivos no relacionados sigan propagándose.
Escrito por el modelo de indexación a partir del texto del issue.
Evaluación
- Stack tecnológico
- cpp, python
- Área
- database, testing
- Tipo de issue
- Error
- Dificultad
- 4/5
- Tiempo estimado
- 3-5 días
- Estado de actividad
- Activo
- Claridad
- Bastante claro
- Aptitud para principiantes
- 68/100