apache / apache/datafusion-python
Report which extension codec handled each node of a serialized plan
- 主要言語
- Python
- スター
- 604
- フォーク
- 174
- 平均マージ
- 1日 7時間
- マージ済み PR(30日)
- 4
説明
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
Extension codecs compose as of #1678: a session holds a chain of them, and encoding walks the chain in install order until one claims an object. With several libraries installed there is currently no way to ask which codec handled a given node. Raised in https://github.com/apache/datafusion-python/pull/1678#pullrequestreview-4940081598.
Part of this is answered already. `SessionContext.logical_extension_codec_ids()` and `physical_extension_codec_ids()` list what is installed, in install order, and those ids are what a payload carries — so a decode failure names the codec that wrote the bytes and lists what the session actually has. What is missing is per-call attribution: which codec handled which node of a particular plan. Today the only way to find out is to check a codec's own call counters before and after, which requires the codec to expose them and tells you nothing about which node was involved.
**Describe the solution you'd like**
Surface the winning codec per node, most naturally in `EXPLAIN` output for a plan that has been serialized, or failing that as an inspection call that reports the encode decisions made for a given plan.
**Describe alternatives you've considered**
Leaving it to the decode error, which already names the responsible codec. That covers the case where something went wrong but not the case where someone is trying to understand a working setup, which is when a multi-library chain is most confusing.
Logging each claim at debug level. Cheap to add and much weaker: it is per-session rather than per-plan, and it puts the burden of correlating lines with nodes on the reader.
**Additional context**
Deliberately left out of #1678: recording the winner means threading it through plan formatting, which is a change to how plans are displayed rather than to how codecs compose.
コントリビューションガイド
このリポジトリのコントリビューションガイドは索引されていません
調査の方向性
Start with SessionContext.logical_extension_codec_ids() and physical_extension_codec_ids(), then read the extension codec chain described in #1678. Determine how a serialized plan is represented for EXPLAIN and define how the winning codec can be attributed to each node; done means a plan-specific inspection or EXPLAIN result identifies the codec handling every serialized node.
索引モデルが issue の本文から書いたものです。
評価
- 技術スタック
- python
- 領域
- backend-api-design, data-engineering
- issue の種類
- 機能追加
- 難易度
- 5/5
- 見積もり時間
- 1週間以上
- 活発さ
- 活発
- 明瞭さ
- おおむね明確
- 初心者へのやさしさ
- 35/100