[PERF] looping through dataframe is 100x slower than when running without cudf
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
**Describe the bug**
I have a case where I loop through each element in a dataframe and call a function for each element. When running with cudf.pandas, this takes on the order of 100x longer time than when running with just pandas. I recognize that best practices is to write vectorized functions but there are cases where it is just easier to loop through each element. I don't expect speedup compared to the non-cudf implementation but it would be good if there wasn't a huge slowdown.
**Steps/Code to reproduce bug**
Code run in a Jupyter notebook:
```
%load_ext cudf.pandas
import pandas as pd
import numpy as np
matrix = np.zeros((100, 100))
df = pd.DataFrame(matrix)
%%time
def func(acc, val):
acc += val
return acc
acc = 0.0
for col in df.columns:
for idx in df.index:
val = df[col][idx]
acc = func(acc, val)
print(acc)
```
**Expected behavior**
When running without cudf this takes 60ms. When running with cudf it takes 10 seconds. I would expect performance with cudf to be comparable to performance without cudf.
**Environment overview (please complete the following information)**
-Bare-metal
-PIP install
**Environment details**
Not sure where to find that script. Here are my basic setup:
Platform: x86 + A100 GPU. Ubuntu 22.04.4 LTS
cuDF: Name: cudf-cu12 Version: 24.6.1
CUDA: Cuda compilation tools, release 12.3, V12.3.107
Python: Python 3.10.12
Running in a Jupyter notebook
**Additional context**
Add any other context about the problem here.
Contributor guide
Assessment
This issue has not been assessed yet.