Serializing custom `TableSource` implementations fails
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Describe the bug
`LogicalExtensionCodec` allows providing a custom serialization strategy for a `TableProvider`, but the calling code always expects a `DefaultTableSource` to unwrap:
https://github.com/apache/datafusion/blob/8d7b11b1cdf04aa9e7c402b5c97e45240e6e71a0/datafusion/proto/src/logical_plan/mod.rs#L1004
Which calls `source_as_provider`:
https://github.com/apache/datafusion/blob/8d7b11b1cdf04aa9e7c402b5c97e45240e6e71a0/datafusion/catalog/src/default_table_source.rs#L99-L110
It would be better if `LogicalExtensionCodec` had `encode_table_source` instead of `encode_table_provider`.
### To Reproduce
_No response_
### Expected behavior
_No response_
### Additional context
_No response_
Contributor guide
Research direction
Start in datafusion/proto/src/logical_plan/mod.rs around line 1004 and trace the LogicalExtensionCodec call into datafusion/catalog/src/default_table_source.rs lines 99-110. Confirm how custom TableSource implementations reach source_as_provider, then verify that serialization no longer requires a DefaultTableSource unwrap and that the relevant proto or catalog tests cover the custom case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100