uutils / uutils/coreutils

Bug: `uu-sort` fails to reorder binary lines containing NUL bytes

Open
#9,264 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

U - sort
Dominant language
Rust
Stars
24.1k
Forks
2k
Avg merge
1d 5h
Merged PRs (30d)
365

Description

Summary

  • Tool under test: uu-sort 0.3.0
  • Impact: When the input stream contains embedded NUL (0x00) and other non-UTF-8 bytes, uu-sort reports success but leaves the data in its original order, whereas GNU sort correctly sorts the lines by their raw byte values.
  • Reproducer: Feed the two-line blob below (first line contains 0x00) into both binaries.
# emit the test case with Python so the NUL byte is preserved
python3 - <<'PY' | /usr/bin/sort | hexdump -C
import sys
sys.stdout.buffer.write(bytes([
    0x84, 0x64, 0x00, 0x48, 0x48, 0x48, 0x48, 0x84, 0xf5, 0x0a,
    0xf5, 0xf5, 0xd9, 0x0b, 0xdd, 0xf5, 0xf5, 0xf5, 0xf5,
]))
PY

Expected result (GNU sort)

$ python3 - <<'PY' | /usr/bin/sort | hexdump -C
> import sys
> sys.stdout.buffer.write(bytes([
>     0x84, 0x64, 0x00, 0x48, 0x48, 0x48, 0x48, 0x84, 0xf5, 0x0a,
>     0xf5, 0xf5, 0xd9, 0x0b, 0xdd, 0xf5, 0xf5, 0xf5, 0xf5,
> ]))
> PY
00000000  f5 f5 d9 0b dd f5 f5 f5  f5 0a 84 64 00 48 48 48  |...........d.HHH|
00000010  48 84 f5 0a                                       |H...|
$ echo $?
0

Actual result (uu-sort)

$ python3 - <<'PY' | instrumented-coreutils/target/release/sort | hexdump -C
> import sys
> sys.stdout.buffer.write(bytes([
>     0x84, 0x64, 0x00, 0x48, 0x48, 0x48, 0x48, 0x84, 0xf5, 0x0a,
>     0xf5, 0xf5, 0xd9, 0x0b, 0xdd, 0xf5, 0xf5, 0xf5, 0xf5,
> ]))
> PY
00000000  84 64 00 48 48 48 48 84  f5 0a f5 f5 d9 0b dd f5  |.d.HHHH.........|
00000010  f5 f5 f5 0a                                       |....|
$ echo $?
0

uu-sort echoes the input verbatim (aside from appending a trailing newline) yet still returns success, so scripts that expect sorted output silently receive unsorted data.

Notes

  • The issue shows up whenever a line includes a NUL and other invalid UTF-8 bytes: uu-sort likely decodes the data as UTF-8 (or via String::from_utf8_lossy) before comparing lines, causing both lines to collapse to the same comparison key and leaving the original order unchanged.
  • Fix idea: compare raw u8 slices rather than UTF-8 Strings so binary inputs remain sortable.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the uu-sort entry point and reproduce the two-line binary input shown in the issue, comparing its output with GNU sort. Trace how input lines become comparison keys; done means embedded NUL and non-UTF-8 bytes are ordered by their raw byte values, with a regression test for the reproducer.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
cli
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.