mapbox / mapbox/mapbox-gl-js

text-transform is not locale-aware

Open
#3,999 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

cross-platform :tv: feature :green_apple:
Dominant language
TypeScript
Stars
12.4k
Forks
2.4k
PR merge metrics
No merged PRs in 30d

Description

Migrated from https://github.com/mapbox/DEPRECATED-mapbox-gl/issues/21 by @1ec5

> The `text-transform` property is [documented in the style specification](https://www.mapbox.com/mapbox-gl-style-spec/#layout-symbol-text-transform) as being “similar to the [CSS `text-transform` property](https://developer.mozilla.org/en-US/docs/Web/CSS/text-transform)”. One key difference is that most modern browser engines transform a text node based on the node’s declared or inherited locale, via the `lang` HTML attribute or `xml:lang` XML attribute, taking into account any language-specific case rules. By contrast, Mapbox GL implementations perform a locale-neutral transformation (for example the “C locale” on POSIX platforms).
>
> A locale-neutral transformation works well for many alphabets, such as English and Spanish, and as expected it has no effect on ideographic writing systems such as the CJK scripts. However, many Latin alphabets have special cases that the C locale doesn’t respect. For example, the Turkish city [Kırşehir](https://en.wikipedia.org/wiki/K%C4%B1r%C5%9Fehir), whose name includes both [dotted and dotless I’s](https://en.wikipedia.org/wiki/Dotted_and_dotless_I), should be labeled “KIRŞEHİR” but instead is labeled “KIRŞEHIR” (omitting a [tittle](https://en.wikipedia.org/wiki/Tittle)):
>
> ![Kırşehir](https://cloud.githubusercontent.com/assets/1231218/17072579/513999f8-501e-11e6-9090-581e6e2e8408.png)
>
> German street names should be labeled, e.g., “GROSSER STERN” instead of “GROßER STERN”:
>
> Großer Stern
>
> It isn’t sufficient to transform text to the user’s current locale. The examples above come from applying `"text-field": "{name}"` in the Bright style. The `name` field in the Mapbox Streets source is written in each feature’s native language, but it provides no way to distinguish between different languages. One could imagine a future version of the source providing a best guess of the name’s language, expressed as a BCP 47 / ISO 639 tag, based on the containing country and some character range–based heuristics. The style specification, then, could be extended with a `text-language` property that would be set to `{language}` for any layer that sets `text-field` to `{name}`.
>
> Adding a `text-language` property isn’t semantically ideal, since it’s really the data that has an intrinsic language, not the style. But it seems like overkill to extend the vector tile specification with a new type that pairs a string with a language identifier.
>
> The native platforms supported by Mapbox GL have standard APIs for uppercasing or lowercasing a string based on a locale. For example, [the Mapbox iOS/macOS SDK implementation](https://github.com/mapbox/mapbox-gl-native/blob/4b1eeada9eaa416585b8e908265240c409c1368f/platform/darwin/src/string_nsstring.mm#L8-28) of `"text-transform": "uppercase"` calls `-[NSString uppercaseString]`, but it should call `-[NSString uppercaseStringWithLocale:]` instead.
>
> On the other hand, Mapbox GL JS calls `String.prototype.toUpperCase()`, and there is currently no standard API for locale-aware conversions beyond the user’s current locale. From the discussion at https://github.com/mapbox/mapbox-gl-js/issues/149#issuecomment-45789708, it sounds like it’d be impractical to include a JavaScript library for pan-language support. However, maybe there’s room to support a handful of high-priority languages like German and Turkish.
>
> The specification should make it clear that locale awareness is made on a best-effort basis, just like in CSS. For example, the Mapbox iOS and macOS SDKs won’t necessarily uppercase the English “E MacDonald St” as “E MacDONALD ST”, the Mapbox Android SDK may fall back to the C locale for Klingon, and Mapbox GL JS wouldn’t be required to do anything differently than it already does.
>
> Beyond text transformations, Mapbox GL could in the future use the `text-language` property to choose the correct national language variant for each [Unihan character](https://en.wikipedia.org/wiki/Han_unification) in CJK text, just as native text rendering engines and Web browser rendering engines do.

cc @mapbox/gl @mapbox/cartography-cats @kkaefer @jfirebaugh

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the Mapbox GL JS implementation of the style specification’s text-transform property and its current String.prototype.toUpperCase() or toLowerCase() entry point. Review the linked discussion about locale-aware conversion and define a best-effort approach for language-specific casing, with corresponding specification and test coverage; done means the scope and fallback behavior are agreed.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
internationalization
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.