python-hyper / python-hyper/rfc3986

To encode or not encode - best practices for "uncommon" uri characters, including whitespaces (%20)?

Open
#104 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
189
Forks
38
Avg merge
13h 36m
Merged PRs (30d)
1

Description

Dear all,

I am currently struggeling with a whitespace problem which I guess should not be that complicated - so I probably missing something here.

MWE:

import rfc3986.builder
rfc3986.builder.URIBuilder.from_uri("scheme:").extend_path("path 1").extend_path("path2").geturl()
# outout: 'scheme:/path 1/path2'
rfc3986.builder.URIBuilder.from_uri("scheme:").extend_path("path 1/path2").geturl()
# outout: 'scheme:/path 1/path2'
rfc3986.builder.URIBuilder.from_uri("scheme:path 1").extend_path("path2").geturl()
# outout: 'scheme:/path%201/path2'

therefore: If i am having a whitespace in the from_uri-part, it gets escaped by %20, whereby having the whitespace as part of the parameter to extend_path, it gets used as is.

From the broader scope, I am storing URIs in a database which get constructed on one component "from scratch" (containing whitespaces ...), whereas they are passed in a url-encoded - conformant manner in another component.
I already figured out that there is an equivalence when passing maybe-url-encoded strings to from_uri:

from_uri_a=rfc3986.builder.URIBuilder.from_uri("scheme:/path 1/path2").finalize()
from_uri_b=rfc3986.builder.URIBuilder.from_uri("scheme:/path%201/path2").finalize()
from_uri_a == from_uri_b
# is True

My main goal is to store the URIs in a future-proof way in my database and from the requirements I am having it does not really make a big difference whether or not I am storing the URLs encoded or not - but from the broader scope I am unsure whether the current implementation is desired or not (aka. a bug or a feature).

From the rfc, sec. 2.4, I guess that an encoding should take place in the extend_path method:

Under normal circumstances, the only time when octets within a URI
are percent-encoded is during the process of producing the URI from
its component parts. This is when an implementation determines which
of the reserved characters are to be used as subcomponent delimiters
and which can be safely used as data. Once produced, a URI is always
in its percent-encoded form.

Any thoughts on this?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the examples using rfc3986.builder.URIBuilder.from_uri, extend_path, and finalize, comparing whitespace in the initial URI with whitespace passed to extend_path. Check the RFC 3986 section linked in the issue and determine whether both construction paths should produce the same percent-encoded result; done means the intended behavior is decided and covered by a regression test.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.