uutils / uutils/coreutils

Inconsistency between tr and cut on multibyte character

Open
#7,600 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

U - tr
Dominant language
Rust
Stars
24.1k
Forks
2k
Avg merge
1d 5h
Merged PRs (30d)
365

Description

Hi,

I downloaded the last release and have a strange behaviour.

I have the following test.txt file (utf8 encoded)

zone1§zone2§zone3
$ ./coreutils cut -d§ -f2 test.txt
zone2
$ cat test.txt | ./coreutils tr '§' ';'
zone1;;zone2;;zone3

the cut command behave as I would expect (although the output is different from the gnu cut)
However, tr should not double ;

Basically, it seems to work fine when tr replace n byte(s) character by another n byte(s) character, but is broken if I try to replace a n byte(s) character by a m byte(s) character.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the reported behavior through the tr command entry point with the UTF-8 test.txt content containing §, then compare its output with cut and the expected single-character replacement. The issue is done when replacing the multibyte character with ; produces one ; per occurrence rather than duplicates, with the behavior covered by a regression test if the relevant test location is found.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
cli
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.