apache / apache/arrow-rs-object-store

Investigate passthrough for `parse_url` on Builders

Open
#793 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
322
Forks
212
Avg merge
5d 2h
Merged PRs (30d)
10

Description

Please proceed with this one! The change is reasonable since the "dual-stack endpoint" conforms to a pre-defined shape.

I was pointing out that `parse_url` currently rejects urls that does not conform to the pre-defined shapes. To support "dual-stack endpoint", we have to add it as a approved shape. Alternatively, we can allow `parse_url` to passthrough any given `url` it does not recognize and set `self.endpoint` accordingly. This way, perhaps we did not need to add the special logic for "dual-stack endpoint" and can also future proof any other url format AWS decides to create.

This was an idea based on #711 where we once again had to allowlist a pre-defined shape into the url parsing logic for Azure/OneLake.

_Originally posted by @kevinjqliu in https://github.com/apache/arrow-rs-object-store/issues/780#issuecomment-4918510487_

Contributor guide

Open the contributing guide

Research direction

Start by tracing `parse_url` in the Builders and how it assigns `self.endpoint`; compare the existing handling with the dual-stack endpoint and the Azure/OneLake precedent in #711. Clarify whether unknown URL shapes should pass through or be explicitly approved, then verify that existing recognized URL formats still behave correctly.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
cloud
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.