duckdb / duckdb/duckdb-python

Surface engine WARNING-level log messages in the Python client (parity with the CLI)

Ouverte
#480 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
Langage dominant
Python
Étoiles
186
Forks
113
Merge moyen
13 h 29 min
PR mergées (30 j)
17

Description

I'd like to suggest a small feature that would bring the Python client closer to parity with the DuckDB CLI.

### Background

The CLI shell already surfaces engine log messages to the user. At startup it registers a custom `LogStorage` (`ShellLogStorage`) and does roughly:

```cpp
log_manager.RegisterLogStorage("shell_log_storage", storage_ptr);
log_manager.SetLogStorage(*db_instance, "shell_log_storage");
log_manager.SetEnableLogging(db_instance);
log_manager.SetLogLevel(duckdb::LogLevel::LOG_WARNING);
```

So a CLI user automatically sees `WARNING`-level messages (e.g. deprecated-syntax notices, the macOS Rosetta perf warning, GEOMETRY/CRS storage-version warnings) printed to the console.

The Python client doesn't do anything analogous — engine logging is left at its defaults (disabled, `memory` storage), so these warnings are effectively invisible to Python/Jupyter users unless they manually `SET enable_logging=true` and `SELECT * FROM duckdb_logs`. The net effect is that **deprecation warnings the CLI shows are silently dropped in Python/Jupyter.**

### Proposal

Add an (opt-in) Python `LogStorage` that forwards engine log entries to Python — ideally via the standard `logging` module, e.g. `logging.getLogger("duckdb").warning(message)` — so users get visibility through machinery they already control (handlers, levels, filters), and notebook users see them inline.

### Prior art in this repo

There's already a clean precedent: the progress bar registers a custom display through `ClientConfig::display_create_func` (`JupyterProgressBarDisplay` in `src/duckdb_py/jupyter/`). A log sink would follow the same shape — a `LogStorage` subclass registered at connection time, alongside where the progress bar is wired up in `SetDefaultConfigArguments()`.

### Implementation notes / care points

- **GIL:** log callbacks fire from executor threads with the GIL released during query execution, so the sink must `py::gil_scoped_acquire` before touching Python — exactly what `JupyterProgressBarDisplay::Update()` already does.
- **Default off / opt-in:** to avoid changing default output (which could disrupt output-diffing test harnesses like `nbval`, or add nondeterministic interleaving), this is probably best behind a connection flag, defaulting off — or at most on only in interactive sessions.
- **Routing through `logging`** (rather than raw stdout/stderr like the CLI) keeps it un-surprising and easily silenced.

### Why it's low-risk

WARNING-level emission is very sparse in the engine today (a handful of call sites, mostly deprecation notices), so the practical noise is minimal — but those are exactly the messages users most benefit from seeing.

Guide de contribution

Ouvrir le guide de contribution

Piste de recherche

Commencez par lire le câblage de la barre de progression dans src/duckdb_py/jupyter/ et SetDefaultConfigArguments(), en particulier JupyterProgressBarDisplay::Update(), puis comparez-le avec l’enregistrement de LogStorage de la CLI décrit dans l’issue. Le travail est terminé lorsqu’une connexion Python opt-in achemine les entrées WARNING du moteur via Python logging sans modifier la sortie par défaut et gère les callbacks provenant des executor threads en toute sécurité.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Évaluation

Stack technique
cpp, python
Domaine
databases
Type d'issue
Fonctionnalité
Difficulté
4/5
Temps estimé
3-5 jours
Activité
Calme
Clarté
Plutôt claire
Accessibilité débutants
48/100

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.