internetarchive / internetarchive/openlibrary
Some URLs containing a readable slug and a trailing `/` character cause unhandled internal errors when visited
- Dominant language
- Python
- Stars
- 6.7k
- Forks
- 2k
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 138
Description
### Problem
Some URLs, when visited, cause the site to serve an unstyled "Internal Server Error" page. This is happening for author, edition, and work URLs that have a trailing `/` character following a readable slug that contains non-latin characters.
#### Evidence / Screenshot
#### Relevant URL(s)
Working URL:
https://openlibrary.org/books/OL4210801M/Das_schwerho%CC%88rige_Kind
Broken URL:
https://openlibrary.org/books/OL4210801M/Das_schwerho%CC%88rige_Kind/
### Reproducing the bug
1. Go to https://openlibrary.org/books/OL4210801M/Das_schwerho%CC%88rige_Kind/
* Expected behavior: You are looking at a book page
* Actual behavior: You are looking at an error message
### Context
- Browser (Chrome, Safari, Firefox, etc): Any
- OS (Windows, Mac, etc): Any
- Logged in (Y/N): N
- Environment (prod, dev, local): prod
Related to this Sentry [issue](https://sentry.archive.org/organizations/ia-ux/issues/42549/?query=is%3Aunresolved&referrer=issue-stream)
### Notes from this Issue's Lead
#### Proposal & constraints
We suspect that this error may have something to do with how our ReadableUrlProcessor is encoding the readable slug and redirecting.
A [similar issue](https://github.com/internetarchive/openlibrary/issues/4311) was solved by creating a [`safe_seeother`](https://github.com/internetarchive/openlibrary/blob/3d1b5ae890f1c4a93d9d431addd442a1f16e9960/openlibrary/plugins/upstream/addbook.py#L179) function, which encodes a URL using [`encode_url_path`](https://github.com/internetarchive/openlibrary/blob/3d1b5ae890f1c4a93d9d431addd442a1f16e9960/openlibrary/plugins/upstream/addbook.py#L160) before passing it to `web.seeother`.
Maybe `encode_url_path` can be used by the `ReadableUrlProcessor` before any redirects?
#### Related files
Code changes likely only needed in this method:
https://github.com/internetarchive/openlibrary/blob/bee8ab4e369a500e1e31f73d3f06fb88e7c5896f/openlibrary/core/processors/readableurls.py#L40
#### Stakeholders
@cdrini
Contributor guide
Research direction
Start in openlibrary/core/processors/readableurls.py at the method identified in the issue, then reproduce the failing URL with its trailing slash and non-Latin slug. Compare its redirect handling with encode_url_path and safe_seeother in the linked addbook.py code; done means the example URL and equivalent author, edition, and work URLs load without an internal error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100