apache / apache/arrow-rs-object-store

Azure abfss Paths can't be parsed in private MS Fabric Environment

Open
#474 3 comments 0 reactions 0 assignees View on GitHub
enhancement good first issue help wanted
Dominant language
Rust
Stars
322
Forks
212
Avg merge
5d 2h
Merged PRs (30d)
10

Description

**Describe the bug**
I'm using https://github.com/delta-io/delta-rs for reading and writing to Microsoft Fabric Onelake, but the error I'm getting points to the issue being part of this repository.
I'm using a fabric workspace with public access disables, when trying to use the normal abfss paths in the style of
``abfs[s]://@onelake.dfs.fabric.microsoft.com/``
I'm getting the error message that public access is disabled.
According to Microsoft [documentation](https://learn.microsoft.com/en-us/fabric/security/security-workspace-level-private-links-overview#connecting-to-workspaces) there are special endpoints available for private link environments, which would correspond to an abfss path like this:
``abfs[s]://@.z.dfs.fabric.microsoft.com/``, where zy are the first to symbols of the workspace id
Trying to access data with this schema yields the error
``Generic MicrosoftAzure error
↳ URL did not match any known pattern for scheme``
Trying to access data with this abfss schema works using other libraries so it should be a parsing issue

**To Reproduce**
Setup ms fabric workspace with public access disabled, then try to access data using the mentioned private link urls.

**Expected behavior**
Paths get matched correctly, and data can be accessed.

Contributor guide

Open the contributing guide

Research direction

No file or test is named in the report. Start by reproducing the failure with the private-link abfss URL in an environment with public access disabled, then trace the URL parsing path and compare it with the working standard abfss form. Done means the private endpoint pattern is accepted and the data can be accessed.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, rust
Domain
cloud
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.