should we normalize scheme and hostname of URI to be lower?
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 483
Description
datasets are defined by their URI
we don't really spelly out what exactly our URI standard is, apart from reserving the `airflow` scheme and ascii
for websites, the scheme and hostname are case insensitive, while everything else is not.
should we normalize scheme and hostname or allow case sensitive differentiation?
my inclination is, we should allow everything to be case sensitive, except _possibly_ the scheme. and the reason is, we don't know exactly what "hostname" will mean for a dataset. if, for example, it's a database object, it could be case sensitive. scheme.
if we do implement some normalization, we introduce somewhat of a problem when we do sqlalchemyf lookups by URI because we can't just do `Dataset.uri == uri`; we'd have to normalize the incoming URI value first. One possibility is to split out the scheme into a different column in the db that has CI collation, which would avoid this issue, though it would force you to decompose. but this is messy too.
Contributor guide
Research direction
Start by reviewing Dataset URI handling and the SQLAlchemy lookup described in the issue. No file or test is named; first locate the current URI equality path and determine how normalization would affect database lookups. Done means the URI standard and case-sensitivity policy are decided, with the resulting behavior covered in the relevant implementation and tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, sqlalchemy
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100