internetarchive / internetarchive/openlibrary

Use Babel to select correct language from the user’s browser settings

Open
#12,102 9 comments 0 reactions 0 assignees View on GitHub
Lead: @mekarpeles Priority: 3 Theme: Internationalization (i18n) Type: Feature Request
Dominant language
Python
Stars
6.7k
Forks
2k
Avg merge
2d 19h
Merged PRs (30d)
138

Description

### Feature Request

## Problem / Opportunity

There are (at least) three different mappings which need to be made, independently, from the user's browser's language settings, taking into account their language preferences in priority order:
1. Book text language - #12003 covers this.
2. UI localization language for Babel - this should use Babel's [`negotiate_locale`](https://babel.readthedocs.io/en/stable/api/core.html?highlight=negotiate_locale#babel.core.negotiate_locale) to find the best match
3. Preferred Wikipedia link - I suspect Wikipedia has an algorithm / lookup to do this, but it would require investigation

As an example, consider zh-HK and zh-TW:

BCP 47 code: `zh-HK`
Book language codes: cmn + chi (`chi` is a macro language code for Chinese, while `cmn` refers to Mandarin Chinese)
Wikipedia code: zh (Chinese Wikipedia)

BCP 47 code: zh-TW
Book language codes: yue + chi
Wikipedia code: zh-yue? (Cantonese Wikipedia is much smaller than Chinese, so zh *might* be a better recommendation)

Unfortunately, the current implementation conflates these three mappings. It also only considers the first language in the Accept-Language header, when the user may have a localization that's supported and preferred to English as their second or third choice. eg. pt-BR, pt, es-419, en

## Proposal

The full browser Accept-Language info should be retained for use in these mappings and for future uses like identifying locales which share a common language across multiple nations, such as Portugese Brazilian (`pt-BR`) or Latin American Spanish (`es-419`).

Babel's Locale.negotiate_locale should be used to find best matches so that more than the first locale/language is considered.

### Breakdown

Implementation Details (for maintainers)

#### Related files

Refer to [this map of common Endpoints](https://docs.openlibrary.org/developers/backend/endpoints.html):
*

#### Requirements Checklist
Checklist of requirements that need to be satisfied in order for this issue to be closed:
* [ ]

#### Stakeholders

*


#### Instructions for Contributors

* **Before** [creating a new branch](https://docs.openlibrary.org/developers/tools/git.html#making-changes-and-creating-a-pull-request) or pushing up changes to a PR, please first [run these commands](https://docs.openlibrary.org/developers/tools/git.html#working-on-your-branch) to ensure your repository is up to date, as the pre-commit bot may add commits to your PRs upstream.

Contributor guide

Open the contributing guide

Research direction

Start with the common Endpoints map linked in the issue and review Babel's negotiate_locale documentation. Trace where the Accept-Language header is currently reduced to one locale, then define separate handling for book language, Babel UI localization, and Wikipedia links. Done means the full preference list is retained and supported non-English choices can be selected independently.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
internationalization
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.