NVIDIA / NVIDIA/cudf

[QST] How'd this work with multi-threading pandas ?

Open
#14,659 2 comments 0 reactions 0 assignees View on GitHub
Python question
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**What is your question?**
I have a multi-thread pandas process to divide large dataset by date ranges then combine them, trying cudf now for speed .

It does something like this , args contains the start/end_dt for each segment, run to process segments
while Pool() as pool :
result = pool.starmap(run, args)

However , running in cudf it threw this error just now.
__pickle.PicklingError: Can't pickle : it's not the same object as pandas._libs.tslibs.timedeltas._timedelta_unpickle_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.