[BUG] [cudf_polars] Single cluster creates rapidsmpf objects for each pdsh query iteration
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
**Describe the bug**
Currently, cudf-polars `single` cluster with `rapidsmpf` execution, creates rapidsmpf objects for every query iteration, where are in other clusters (`distributed`, and `spmd`), they are created only once during the full query loop, and reused for every query (with a new `Context` for every query).
**Steps/Code to reproduce bug**
**Expected behavior**
`Single` should follow the same behavior as other clusters.
call `rapidsmpf.intergrations.single.get_worker_context, setup_worker` to access the global worker context and use it with a new Context for each query.
**Environment overview (please complete the following information)**
- Environment location: [Bare-metal, Docker, Cloud(specify cloud provider)]
- Method of cuDF install: [conda, Docker, or from source]
- If method of install is [Docker], provide `docker pull` & `docker run` commands used
**Environment details**
Please run and paste the output of the `cudf/print_env.sh` script here, to gather any other relevant environment details
**Additional context**
Add any other context about the problem here.
Contributor guide
Assessment
This issue has not been assessed yet.