When the amount of data is not small,The parameter Num of the parallel process becomes larger, but the running time becomes longer
Open
0 - Waiting on Author
bug
Python
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
**Describe the bug**
**Steps/Code to reproduce bug**
```
import cudf
from multiprocessing import get_context
import time
pdf = cudf.DataFrame({
'low':[i for i in range(1000)],
'close':[i for i in range(1000)],
})
def get_df(idx):
return idx.rolling(5).mean()
if __name__ == "__main__":
ctx = get_context("spawn")
start = time.time()
num =1
with ctx.Pool(num) as pool:
cudf.concat(pool.map(get_df, [pdf.a[i:] for i in range(100, 200)]))
print(time.time()-start)
```
num =1 ,time 3.2659552097320557
num =2, time 7.382307291030884
Contributor guide
Assessment
This issue has not been assessed yet.