tc39 / tc39/test262

Missing guidelines/patterns for locale-sensitive testing

Open
#3,786 6 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
2.8k
Forks
564
Avg merge
3d 7h
Merged PRs (30d)
10

Description

Nearly all ECMA-402 APIs (and some in ECMA-262) have implementation- and locale-dependent ("ILD") behavior, and those details change over time (e.g. with new browser customizations or CLDR/ICU releases). Relying on "golden output" is therefore brittle, but we also want to achieve good coverage.

There are options such as looking at stable substrings or other parts of output (such as month names), comparative testing across related inputs (e.g. that each setting for an input option produces a distinct output), general metamorphic testing, and broader decisions to make such as the set of locales and locale options to utilize (and whether or not to always utilize all of them).

Guidance should be added to CONTRIBUTING.md in the form of both explanations and examples. A proposed first step in establishing such guidance is to start with a relatively simple ILD API, and Intl.RelativeTimeFormat seems like a good candidate because it supports a small number of options (just numberingSystem, style "long" vs. "short" vs. "narrow", and numeric "always" vs. "auto") and its formatted output includes both numeric and textual parts (the former of which depends upon and can therefore be compared against Intl.NumberFormat functionality). Let's take a look at the existing tests and see what conclusions can be drawn from them.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the existing Intl.RelativeTimeFormat tests and compare their handling of numberingSystem, style, numeric, and locale-dependent output with related Intl.NumberFormat behavior. Record conclusions about stable substrings, comparative or metamorphic checks, and locale selection, then add explanations and examples to CONTRIBUTING.md.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
documentation, internationalization, testing-qa
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.