astropy / astropy/astropy

Utils: return more structured `urllib.request.Request`.

Open
#18,092 9 comments 0 reactions 0 assignees View on GitHub
API change utils
Dominant language
Python
Stars
5.3k
Forks
2.2k
Avg merge
1d 18h
Merged PRs (30d)
74

Description

Flagging for follow-up for replacing the output with the more structured `urllib.parse.ParseResult` or a https://docs.python.org/3/library/urllib.request.html#urllib.request.Request.

_Originally posted by @nstarman in https://github.com/astropy/astropy/pull/18091#discussion_r2071893156_

There's an underlying philosophical point: structurable data should be structured. Especially in a OOPL. URLs aren't strings, though they can be represented by them. Like file paths aren't strings, but can be represented as such. Basically if something requires parsing and validation then it's better to do that once and return a structured output.

This idea and more is explored the excellent linked [here](https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate/) about type-driven design. The TL;DR is that structured objects can make guarantees about their contents. Those guarantees are useful to prevent having to always validate data since the validation checks are true by construction. For example, strings are more general than floats — strings can hold `"3.1415"` and `"a string"` — but we find it useful to make guarantees about mathematical operations by storing numbers in float types. Like `3.14` as a float can be stored in string form, so too can URLs. That doesn't make it a good design, and IMO URLs, file-paths, floats, etc are all best represented as distinct objects, not strings.

We've started on file-paths, with PRs enabling use of `pathlib.Path` objects. We should similarly improve URLs.

Contributor guide

Open the contributing guide

Research direction

No project files or tests are named. Start by locating the utility API whose current output is being reconsidered, then compare returning urllib.parse.ParseResult with urllib.request.Request and identify compatibility requirements. Done means the project has a decided structured URL representation and corresponding tests or migration guidance.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend-api-design
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.