uutils / uutils/coreutils

cp performance issue

Open
#7,092 23 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

I - Performance (Speed) U - cp
Dominant language
Rust
Stars
24.1k
Forks
2k
Avg merge
1d 5h
Merged PRs (30d)
365

Description

As is observed in https://github.com/nushell/nushell/issues/14778, uu_cp seems to have performance issues when dealing with large directories.

> timeit ~/.cargo/bin/coreutils cp cymbol_out/ -r cymbol_out_bak
52sec 95ms 4µs 960ns
> timeit ~/.cargo/bin/coreutils rm -rf cymbol_out_bak
10ms 929µs 36ns
> ~/.cargo/bin/coreutils -h
coreutils 0.0.28 (multi-call binary)
...

In my case, the directory is of ~2.5GB containing a list of big json files. GNU coreutils cp takes <3s to complete the operation but uu_cp takes ~50s. I've also tried on zsh with similar results.

> time ~/.cargo/bin/coreutils cp cymbol_out -r cymbol_out_bak
real    50.43s
user    0.00s
sys     3.23s
> time ~/.cargo/bin/coreutils rm -r cymbol_out_bak
real    0.56s
user    0.00s
sys     0.56s
> time cp cymbol_out -r cymbol_out_bak
real    2.81s
user    0.00s
sys     2.81s
> time rm -r cymbol_out_bak
real    0.54s
user    0.00s
sys     0.54s

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the reported uu_cp versus GNU cp timings with the large directory and JSON files described in the issue. Compare recursive-copy behavior and measure where the time is spent; done means uu_cp no longer shows the reported large-directory slowdown and the benchmark confirms the improvement.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
cli, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.