NVIDIA / NVIDIA/cudf

[FEA] Improve the performance of sample

Open
#9,834 3 comments 0 reactions 0 assignees View on GitHub
0 - Backlog feature request libcudf Performance Spark
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**Is your feature request related to a problem? Please describe.**
The sample without replacement is slower than the CPU.

**Additional context**
With replacement is really fast.
This indicates that thrust::shuffle_copy is the reason for the slowness.
```
// replacement is true
spark.time(spark.range(Int.MaxValue * 12L).sample(true, 0.01, 0).selectExpr("SUM(id)", "COUNT(id)").show())
+-------------------+---------+
| sum(id)|count(id)|
+-------------------+---------+
|3320410258800588221|257697960|
+-------------------+---------+
Time taken: 670 ms

// without replacement
scala> spark.time(spark.range(Int.MaxValue).sample(0.01, 0).agg(functions.sum("id")).show())
+-----------------+
| sum(id)|
+-----------------+
|23058247476802342|
+-----------------+

Time taken: 1608 ms
```

[Sample code link](https://github.com/rapidsai/cudf/blob/branch-22.02/cpp/include/cudf/copying.hpp#L935)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.