Bug: `uu-sort` fails to reorder binary lines containing NUL bytes
Open
Nobody has claimed this yet.
U - sort
- Dominant language
- Rust
- Stars
- 24.1k
- Forks
- 2k
- Avg merge
- 1d 5h
- Merged PRs (30d)
- 365
Description
Summary
- Tool under test:
uu-sort0.3.0 - Impact: When the input stream contains embedded
NUL(0x00) and other non-UTF-8 bytes,uu-sortreports success but leaves the data in its original order, whereas GNUsortcorrectly sorts the lines by their raw byte values. - Reproducer: Feed the two-line blob below (first line contains
0x00) into both binaries.
# emit the test case with Python so the NUL byte is preserved
python3 - <<'PY' | /usr/bin/sort | hexdump -C
import sys
sys.stdout.buffer.write(bytes([
0x84, 0x64, 0x00, 0x48, 0x48, 0x48, 0x48, 0x84, 0xf5, 0x0a,
0xf5, 0xf5, 0xd9, 0x0b, 0xdd, 0xf5, 0xf5, 0xf5, 0xf5,
]))
PY
Expected result (GNU sort)
$ python3 - <<'PY' | /usr/bin/sort | hexdump -C
> import sys
> sys.stdout.buffer.write(bytes([
> 0x84, 0x64, 0x00, 0x48, 0x48, 0x48, 0x48, 0x84, 0xf5, 0x0a,
> 0xf5, 0xf5, 0xd9, 0x0b, 0xdd, 0xf5, 0xf5, 0xf5, 0xf5,
> ]))
> PY
00000000 f5 f5 d9 0b dd f5 f5 f5 f5 0a 84 64 00 48 48 48 |...........d.HHH|
00000010 48 84 f5 0a |H...|
$ echo $?
0
Actual result (uu-sort)
$ python3 - <<'PY' | instrumented-coreutils/target/release/sort | hexdump -C
> import sys
> sys.stdout.buffer.write(bytes([
> 0x84, 0x64, 0x00, 0x48, 0x48, 0x48, 0x48, 0x84, 0xf5, 0x0a,
> 0xf5, 0xf5, 0xd9, 0x0b, 0xdd, 0xf5, 0xf5, 0xf5, 0xf5,
> ]))
> PY
00000000 84 64 00 48 48 48 48 84 f5 0a f5 f5 d9 0b dd f5 |.d.HHHH.........|
00000010 f5 f5 f5 0a |....|
$ echo $?
0
uu-sort echoes the input verbatim (aside from appending a trailing newline) yet still returns success, so scripts that expect sorted output silently receive unsorted data.
Notes
- The issue shows up whenever a line includes a
NULand other invalid UTF-8 bytes:uu-sortlikely decodes the data as UTF-8 (or viaString::from_utf8_lossy) before comparing lines, causing both lines to collapse to the same comparison key and leaving the original order unchanged. - Fix idea: compare raw
u8slices rather than UTF-8Strings so binary inputs remain sortable.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the uu-sort entry point and reproduce the two-line binary input shown in the issue, comparing its output with GNU sort. Trace how input lines become comparison keys; done means embedded NUL and non-UTF-8 bytes are ordered by their raw byte values, with a regression test for the reproducer.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- cli
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100