apache / apache/datafusion-python

Report which extension codec handled each node of a serialized plan

未關閉
#1,706 0 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
enhancement
主要語言
Python
星號
604
分支
174
平均合併
2 天 22 小時
30 天內合併 PR
5

描述

**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**

Extension codecs compose as of #1678: a session holds a chain of them, and encoding walks the chain in install order until one claims an object. With several libraries installed there is currently no way to ask which codec handled a given node. Raised in https://github.com/apache/datafusion-python/pull/1678#pullrequestreview-4940081598.

Part of this is answered already. `SessionContext.logical_extension_codec_ids()` and `physical_extension_codec_ids()` list what is installed, in install order, and those ids are what a payload carries — so a decode failure names the codec that wrote the bytes and lists what the session actually has. What is missing is per-call attribution: which codec handled which node of a particular plan. Today the only way to find out is to check a codec's own call counters before and after, which requires the codec to expose them and tells you nothing about which node was involved.

**Describe the solution you'd like**

Surface the winning codec per node, most naturally in `EXPLAIN` output for a plan that has been serialized, or failing that as an inspection call that reports the encode decisions made for a given plan.

**Describe alternatives you've considered**

Leaving it to the decode error, which already names the responsible codec. That covers the case where something went wrong but not the case where someone is trying to understand a working setup, which is when a multi-library chain is most confusing.

Logging each claim at debug level. Cheap to add and much weaker: it is per-session rather than per-plan, and it puts the burden of correlating lines with nodes on the reader.

**Additional context**

Deliberately left out of #1678: recording the winner means threading it through plan formatting, which is a change to how plans are displayed rather than to how codecs compose.

貢獻指南

這個儲存庫沒有索引到貢獻指南

研究方向

Start with SessionContext.logical_extension_codec_ids() and physical_extension_codec_ids(), then read the extension codec chain described in #1678. Determine how a serialized plan is represented for EXPLAIN and define how the winning codec can be attributed to each node; done means a plan-specific inspection or EXPLAIN result identifies the codec handling every serialized node.

由索引模型根據 Issue 內容生成。

評估

技術堆疊
python
領域
backend-api-design, data-engineering
Issue 類型
功能
難度
5/5
預估耗時
一週以上
活躍度
活躍
描述清晰度
基本清楚
新手友好度
35/100

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。