internetarchive / internetarchive/openlibrary

Some URLs containing a readable slug and a trailing `/` character cause unhandled internal errors when visited

Open
#8,990 5 comments 0 reactions 0 assignees View on GitHub
Lead: @jimchamp Needs: Investigation Priority: 4 Type: Bug
Dominant language
Python
Stars
6.7k
Forks
2k
Avg merge
2d 19h
Merged PRs (30d)
138

Description

### Problem

Some URLs, when visited, cause the site to serve an unstyled "Internal Server Error" page. This is happening for author, edition, and work URLs that have a trailing `/` character following a readable slug that contains non-latin characters.

#### Evidence / Screenshot

#### Relevant URL(s)

Working URL:
https://openlibrary.org/books/OL4210801M/Das_schwerho%CC%88rige_Kind

Broken URL:
https://openlibrary.org/books/OL4210801M/Das_schwerho%CC%88rige_Kind/

### Reproducing the bug

1. Go to https://openlibrary.org/books/OL4210801M/Das_schwerho%CC%88rige_Kind/

* Expected behavior: You are looking at a book page
* Actual behavior: You are looking at an error message

### Context

- Browser (Chrome, Safari, Firefox, etc): Any
- OS (Windows, Mac, etc): Any
- Logged in (Y/N): N
- Environment (prod, dev, local): prod

Related to this Sentry [issue](https://sentry.archive.org/organizations/ia-ux/issues/42549/?query=is%3Aunresolved&referrer=issue-stream)

### Notes from this Issue's Lead

#### Proposal & constraints

We suspect that this error may have something to do with how our ReadableUrlProcessor is encoding the readable slug and redirecting.

A [similar issue](https://github.com/internetarchive/openlibrary/issues/4311) was solved by creating a [`safe_seeother`](https://github.com/internetarchive/openlibrary/blob/3d1b5ae890f1c4a93d9d431addd442a1f16e9960/openlibrary/plugins/upstream/addbook.py#L179) function, which encodes a URL using [`encode_url_path`](https://github.com/internetarchive/openlibrary/blob/3d1b5ae890f1c4a93d9d431addd442a1f16e9960/openlibrary/plugins/upstream/addbook.py#L160) before passing it to `web.seeother`.

Maybe `encode_url_path` can be used by the `ReadableUrlProcessor` before any redirects?

#### Related files

Code changes likely only needed in this method:
https://github.com/internetarchive/openlibrary/blob/bee8ab4e369a500e1e31f73d3f06fb88e7c5896f/openlibrary/core/processors/readableurls.py#L40

#### Stakeholders

@cdrini

Contributor guide

Open the contributing guide

Research direction

Start in openlibrary/core/processors/readableurls.py at the method identified in the issue, then reproduce the failing URL with its trailing slash and non-Latin slug. Compare its redirect handling with encode_url_path and safe_seeother in the linked addbook.py code; done means the example URL and equivalent author, edition, and work URLs load without an internal error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.