sort is not conformant with POSIX (collation, i18n environment variables)
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 24.1k
- Forks
- 2k
- Avg merge
- 1d 5h
- Merged PRs (30d)
- 365
Description
https://pubs.opengroup.org/onlinepubs/9799919799/utilities/sort.html#tag_20_112_08
The following environment variables shall affect the execution of sort:
- LANG
- Provide a default value for the internationalization variables that are unset or null. (See XBD 8.2 Internationalization Variables for the precedence of internationalization variables used to determine the values of locale categories.)
- LC_ALL
- If set to a non-empty string value, override the values of all the other internationalization variables.
- LC_COLLATE
Determine the locale for ordering rules.
Under Hungarian locale, sort result is expected as follows:
$ printf '%s\n' g{x..z} | sort
gx
gz
gy
I tested GNU, BSD, and uutils sort’s under product of LANG, LC_ALL, and LC_COLLATE where
- unset
- set to empty
- set to
en_US.UTF-8 - set to
hu_HU.UTF-8
GNU and BSD behave as the same, in conformity to POSIX, but uu-sort e939eda7bdea0391161eb99687e8a8a7054671bd does not give correct ordering under the following conditions:
LANG |
LC_ALL |
LC_COLLATE |
|---|---|---|
| unset | empty | hu_HU.UTF-8 |
| empty | empty | hu_HU.UTF-8 |
en_US.UTF-8 |
empty | hu_HU.UTF-8 |
hu_HU.UTF-8 |
unset | empty |
hu_HU.UTF-8 |
empty | unset |
hu_HU.UTF-8 |
empty | empty |
hu_HU.UTF-8 |
empty | hu_HU.UTF-8 |
e.g.
$ unset LANG
$ export LC_ALL=
$ export LC_COLLATE=hu_HU.UTF-8
$ printf '%s\n' g{x..z} | gnusort
gx
gz
gy
$ printf '%s\n' g{x..z} | ./target/debug/sort
gx
gy
gz
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the Hungarian-locale examples with the uu-sort binary and compare them with GNU sort across the listed LANG, LC_ALL, and LC_COLLATE combinations. Trace sort's locale and collation handling, then verify that the expected gx, gz, gy ordering is produced for every failing environment combination.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- cli, localization
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100