intake / intake/intake

[Enhancement] Proposal for introducing centralized URI handling

Open
#793 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.1k
Forks
151
Avg merge
8h 24m
Merged PRs (30d)
2

Description

I'm not familiar with the structure of the project, but I have the impression that [there](https://github.com/search?q=repo%3Aintake%2Fintake+%22%5C%22%3A%2F%22&type=code) [are](https://github.com/search?q=repo%3Aintake%2Fintake+%22%5C%22%2F%22+path%3A%2F%5Eintake%5C%2F%2F&type=code) [too](https://github.com/search?q=repo%3Aintake%2Fintake+split%28&type=code) [many](https://github.com/search?q=repo%3Aintake%2Fintake+%22%5C%22%3A%5C%22%22+path%3A%2F%5Eintake%5C%2F%2F&type=code) [places](https://github.com/search?q=repo%3Aintake%2Fintake+%22%5C%22%2F%5C%22%22+path%3A%2F%5Eintake%5C%2F%2F&type=code) where we do URI string parsing, protocol extraction, checking URI format, converting paths to POSIX format, and so on...

It seems tricky to maintain, and it could cause some weird bugs. I think there should be the only place where we do all the operations with URI strings.

For example, it could be done like that, in this very draft implementation of the `URI` class:

```python
from pathlib import Path
from typing import Literal
from dataclasses import dataclass, InitVar, field

Protocols = Literal[
"file",
"sqlite",
"postgresql",
"http",
"https",
"s3",
]

@dataclass
class Credentials:
username: str | None
password: str | None

@dataclass
class URI:
uri: InitVar[str]
path: Path = field(init=False)
protocol: Protocols = field(init=False)
credentials: Credentials = field(init=False)

def __post_init__(self, uri: str):
# Extract protocol
self.protocol = self._get_protocol(uri)
# Extract authoritative notation
self.credentials = self._get_credentials(uri)
# Extract path
self.path = Path(self._get_path(uri))
...

def to_str(self, protocol: bool = True, credentials: bool = True, posix: bool = True) -> str:
if posix:
path = self.path.as_posix()
if protocol:
path = f"{protocol}://{path}"
if credentials:
path = ...
...
return str(path)

def _get_protocol(self, uri: str) -> Protocols:
...
return uri.split(":/")[0]

def _get_credentials(self, uri: str) -> Credentials:
if self.protocol == "file":
...
elif self.protocol == "http":
...
return Credentials(None, None)

def _get_path(self, uri: str) -> Path:
...
return Path(uri)

uri = URI("https://github.com/intake/intake")
if uri.protocol == "file":
open(uri.to_str(protocol=False))
elif uri.protocol == "postgresql":
...
```

Probably, `intake/utils.py` could be the right place for it...

What do you think if we create this missing abstraction layer where we do all the manipulations, and then we just pass the `URI` class instances everywhere?
I think this should simplify the code, ease maintenance, and promote code reuse.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing the URI-parsing call sites linked in the issue and checking whether intake/utils.py is an appropriate entry point. Compare the existing protocol, credential, path, and formatting operations with the proposed URI abstraction; the work is done when the project has an agreed centralized design and its affected callers use it consistently.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.