internetarchive / internetarchive/openlibrary
Use Babel to select correct language from the user’s browser settings
- Dominant language
- Python
- Stars
- 6.7k
- Forks
- 2k
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 138
Description
### Feature Request
## Problem / Opportunity
There are (at least) three different mappings which need to be made, independently, from the user's browser's language settings, taking into account their language preferences in priority order:
1. Book text language - #12003 covers this.
2. UI localization language for Babel - this should use Babel's [`negotiate_locale`](https://babel.readthedocs.io/en/stable/api/core.html?highlight=negotiate_locale#babel.core.negotiate_locale) to find the best match
3. Preferred Wikipedia link - I suspect Wikipedia has an algorithm / lookup to do this, but it would require investigation
As an example, consider zh-HK and zh-TW:
BCP 47 code: `zh-HK`
Book language codes: cmn + chi (`chi` is a macro language code for Chinese, while `cmn` refers to Mandarin Chinese)
Wikipedia code: zh (Chinese Wikipedia)
BCP 47 code: zh-TW
Book language codes: yue + chi
Wikipedia code: zh-yue? (Cantonese Wikipedia is much smaller than Chinese, so zh *might* be a better recommendation)
Unfortunately, the current implementation conflates these three mappings. It also only considers the first language in the Accept-Language header, when the user may have a localization that's supported and preferred to English as their second or third choice. eg. pt-BR, pt, es-419, en
## Proposal
The full browser Accept-Language info should be retained for use in these mappings and for future uses like identifying locales which share a common language across multiple nations, such as Portugese Brazilian (`pt-BR`) or Latin American Spanish (`es-419`).
Babel's Locale.negotiate_locale should be used to find best matches so that more than the first locale/language is considered.
### Breakdown
Implementation Details (for maintainers)
#### Related files
Refer to [this map of common Endpoints](https://docs.openlibrary.org/developers/backend/endpoints.html):
*
#### Requirements Checklist
Checklist of requirements that need to be satisfied in order for this issue to be closed:
* [ ]
#### Stakeholders
*
#### Instructions for Contributors
* **Before** [creating a new branch](https://docs.openlibrary.org/developers/tools/git.html#making-changes-and-creating-a-pull-request) or pushing up changes to a PR, please first [run these commands](https://docs.openlibrary.org/developers/tools/git.html#working-on-your-branch) to ensure your repository is up to date, as the pre-commit bot may add commits to your PRs upstream.
Contributor guide
Research direction
Start with the common Endpoints map linked in the issue and review Babel's negotiate_locale documentation. Trace where the Accept-Language header is currently reduced to one locale, then define separate handling for book language, Babel UI localization, and Wikipedia links. Done means the full preference list is retained and supported non-English choices can be selected independently.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- internationalization
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100