NVIDIA / NVIDIA/cudf

When the amount of data is not small,The parameter Num of the parallel process becomes larger, but the running time becomes longer

Open
#13,549 1 comment 0 reactions 0 assignees View on GitHub
0 - Waiting on Author bug Python
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**Describe the bug**

**Steps/Code to reproduce bug**
```
import cudf
from multiprocessing import get_context
import time
pdf = cudf.DataFrame({
'low':[i for i in range(1000)],
'close':[i for i in range(1000)],
})

def get_df(idx):
return idx.rolling(5).mean()

if __name__ == "__main__":
ctx = get_context("spawn")
start = time.time()
num =1
with ctx.Pool(num) as pool:
cudf.concat(pool.map(get_df, [pdf.a[i:] for i in range(100, 200)]))
print(time.time()-start)

```

num =1 ,time 3.2659552097320557
num =2, time 7.382307291030884

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.