apache / apache/datafusion

ListingTableUrl should allow direct construction

Open
#12,581 2 comments 1 reaction 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Is your feature request related to a problem or challenge?

When investigating delta-io/delta-rs#2834 I discovered that the crux of the problem was that our (delta-rs) code has Urls coming from object_store, which are then passed back into Datafusion via `read_parquet` which does a ListingTableUrl::parse on those strings, thereby turning URLs into strings and "corrupting" them in the process.

### Describe the solution you'd like

I think a `ListingTableUrl` constructor or means of construction without a parse would suffice for our use-case. On the delta-rs side we have effectively all the necessary information to construct a `ListingTableUrl` (url, scheme, object store, etc) but we are not able to build that ourselves and therefore have to jump through a couple hoops to get ListingTableUrl::parse to produce the right thing.

### Describe alternatives you've considered

Right now I'm passing str::replcae(meta.location.as_ref(), "%", "%25") to read_parquet and it feels yucky :laughing: :naus

### Additional context

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by locating ListingTableUrl::parse and the read_parquet path that calls it, then inspect how object_store URLs and schemes are represented. Define completion as providing a direct construction path that preserves URL contents without string replacement, with tests covering the affected URL handling.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend-api-design
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.