apache / apache/arrow-rs-object-store
Azure abfss Paths can't be parsed in private MS Fabric Environment
- Dominant language
- Rust
- Stars
- 322
- Forks
- 212
- Avg merge
- 5d 2h
- Merged PRs (30d)
- 10
Description
**Describe the bug**
I'm using https://github.com/delta-io/delta-rs for reading and writing to Microsoft Fabric Onelake, but the error I'm getting points to the issue being part of this repository.
I'm using a fabric workspace with public access disables, when trying to use the normal abfss paths in the style of
``abfs[s]://@onelake.dfs.fabric.microsoft.com/``
I'm getting the error message that public access is disabled.
According to Microsoft [documentation](https://learn.microsoft.com/en-us/fabric/security/security-workspace-level-private-links-overview#connecting-to-workspaces) there are special endpoints available for private link environments, which would correspond to an abfss path like this:
``abfs[s]://@.z.dfs.fabric.microsoft.com/``, where zy are the first to symbols of the workspace id
Trying to access data with this schema yields the error
``Generic MicrosoftAzure error
↳ URL did not match any known pattern for scheme``
Trying to access data with this abfss schema works using other libraries so it should be a parsing issue
**To Reproduce**
Setup ms fabric workspace with public access disabled, then try to access data using the mentioned private link urls.
**Expected behavior**
Paths get matched correctly, and data can be accessed.
Contributor guide
Research direction
No file or test is named in the report. Start by reproducing the failure with the private-link abfss URL in an environment with public access disabled, then trace the URL parsing path and compare it with the working standard abfss form. Done means the private endpoint pattern is accepted and the data can be accessed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, rust
- Domain
- cloud
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100