apache / apache/datafusion-python

Report which extension codec handled each node of a serialized plan

オープン
#1,706 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る
enhancement
主要言語
Python
スター
604
フォーク
174
平均マージ
1日 7時間
マージ済み PR(30日)
4

説明

**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**

Extension codecs compose as of #1678: a session holds a chain of them, and encoding walks the chain in install order until one claims an object. With several libraries installed there is currently no way to ask which codec handled a given node. Raised in https://github.com/apache/datafusion-python/pull/1678#pullrequestreview-4940081598.

Part of this is answered already. `SessionContext.logical_extension_codec_ids()` and `physical_extension_codec_ids()` list what is installed, in install order, and those ids are what a payload carries — so a decode failure names the codec that wrote the bytes and lists what the session actually has. What is missing is per-call attribution: which codec handled which node of a particular plan. Today the only way to find out is to check a codec's own call counters before and after, which requires the codec to expose them and tells you nothing about which node was involved.

**Describe the solution you'd like**

Surface the winning codec per node, most naturally in `EXPLAIN` output for a plan that has been serialized, or failing that as an inspection call that reports the encode decisions made for a given plan.

**Describe alternatives you've considered**

Leaving it to the decode error, which already names the responsible codec. That covers the case where something went wrong but not the case where someone is trying to understand a working setup, which is when a multi-library chain is most confusing.

Logging each claim at debug level. Cheap to add and much weaker: it is per-session rather than per-plan, and it puts the burden of correlating lines with nodes on the reader.

**Additional context**

Deliberately left out of #1678: recording the winner means threading it through plan formatting, which is a change to how plans are displayed rather than to how codecs compose.

コントリビューションガイド

このリポジトリのコントリビューションガイドは索引されていません

調査の方向性

Start with SessionContext.logical_extension_codec_ids() and physical_extension_codec_ids(), then read the extension codec chain described in #1678. Determine how a serialized plan is represented for EXPLAIN and define how the winning codec can be attributed to each node; done means a plan-specific inspection or EXPLAIN result identifies the codec handling every serialized node.

索引モデルが issue の本文から書いたものです。

評価

技術スタック
python
領域
backend-api-design, data-engineering
issue の種類
機能追加
難易度
5/5
見積もり時間
1週間以上
活発さ
活発
明瞭さ
おおむね明確
初心者へのやさしさ
35/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。