uutils / uutils/coreutils

sort is not conformant with POSIX (collation, i18n environment variables)

Open
#11,136 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

U - sort
Dominant language
Rust
Stars
24.1k
Forks
2k
Avg merge
1d 5h
Merged PRs (30d)
365

Description

https://pubs.opengroup.org/onlinepubs/9799919799/utilities/sort.html#tag_20_112_08

The following environment variables shall affect the execution of sort:

LANG
Provide a default value for the internationalization variables that are unset or null. (See XBD 8.2 Internationalization Variables for the precedence of internationalization variables used to determine the values of locale categories.)
LC_ALL
If set to a non-empty string value, override the values of all the other internationalization variables.
LC_COLLATE

Determine the locale for ordering rules.

Under Hungarian locale, sort result is expected as follows:

$ printf '%s\n' g{x..z} | sort
gx
gz
gy

I tested GNU, BSD, and uutils sort’s under product of LANG, LC_ALL, and LC_COLLATE where

  • unset
  • set to empty
  • set to en_US.UTF-8
  • set to hu_HU.UTF-8

GNU and BSD behave as the same, in conformity to POSIX, but uu-sort e939eda7bdea0391161eb99687e8a8a7054671bd does not give correct ordering under the following conditions:

LANG LC_ALL LC_COLLATE
unset empty hu_HU.UTF-8
empty empty hu_HU.UTF-8
en_US.UTF-8 empty hu_HU.UTF-8
hu_HU.UTF-8 unset empty
hu_HU.UTF-8 empty unset
hu_HU.UTF-8 empty empty
hu_HU.UTF-8 empty hu_HU.UTF-8

e.g.

$ unset LANG
$ export LC_ALL=
$ export LC_COLLATE=hu_HU.UTF-8
$ printf '%s\n' g{x..z} | gnusort
gx
gz
gy
$ printf '%s\n' g{x..z} | ./target/debug/sort
gx
gy
gz

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the Hungarian-locale examples with the uu-sort binary and compare them with GNU sort across the listed LANG, LC_ALL, and LC_COLLATE combinations. Trace sort's locale and collation handling, then verify that the expected gx, gz, gy ordering is produced for every failing environment combination.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
cli, localization
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.