`sort`: Find a way to parallelize number parsing
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
- Issue type
- Refactor
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- rust
- Domain
- cli, performance
Research direction
Start by inspecting chunks.rs, especially parse_lines and its side effects in line_data, then run the profiling setup described with sort -g. The work is done when number parsing is parallelized while preserving correct input order and avoiding a substantial single-thread performance loss.
Written by the indexing model from the issue text.
Description
Especially after #8031 (pending PR), number parsing is a significant component of sort -g runtime (and possibly of other modes as well), maybe around ~70% of the runtime, as that part is single-threaded, while the sorting operation runs on multiple cores.
However, parallelizing chunks.rs parse_lines is challenging as there are side effects in line_data and we still need the data to be in the correct order. This might required extensive refactoring. We should find a way to make it parallel without sacrificing single-thread performance (at least not too much).
Testing setup:
seq 100 -0.000258 0 | sort -R > tmp/floatdata
cargo build --profile profiling -p uu_sort && samply record -r 10000 target/profiling/sort -g tmp/floatdata > /dev/null
- Dominant language
- Rust
- Stars
- 24.1k
- Forks
- 2k
- Avg merge
- 1d 5h
- Merged PRs (30d)
- 365
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from uutils/coreutils
-
U - ls
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
-
U - ls
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
U - tr
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
U - date
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
U - date
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
All issues in uutils/coreutils
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
kwakseongjae/auto-hwp#319 ·
-
area:cli bug filter-quality good first issue priority:medium
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
-
Difficulty 1/5 Under an hour Newbie friendliness 72/100
bevyengine/bevy#25861 ·
-
comp-datalake
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
ClickHouse/ClickHouse#121222 ·
-
enhancement remote
Difficulty 2/5 1-3 hours Newbie friendliness 68/100