NVIDIA-Merlin / NVIDIA-Merlin/Transformers4Rec

[BUG] save_time_based_splits function does not support CPU mode well

Open
#789 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug status/needs-triage
Dominant language
Python
Stars
1.3k
Forks
165
Avg merge
1m
Merged PRs (30d)
2

Description

Bug description

Function save_time_based_splits in data_utils.py does not support CPU mode correctly. In particular, function _save_time_based_splits_cpu assumes using Rapids libraries, moreover Dask Dataframe seems incorrectly imported.

Steps/Code to reproduce bug

Using code from examples, just with option CPU set to True (https://github.com/NVIDIA-Merlin/Transformers4Rec/blob/main/examples/getting-started-session-based/01-ETL-with-NVTabular.ipynb)

sessions_gdf = df.read_parquet(BASE_PATH / "processed_nvt/part_0.parquet")
from transformers4rec.utils.data_utils import save_time_based_splits

save_time_based_splits(
data=nvt.Dataset(sessions_gdf),
output_dir=BASE_PATH / f"session_by_day",
partition_col="day-first",
timestamp_col="session_id",
cpu=True
)

Expected behavior

No exception is thrown and data are splitted.

Environment details
  • Transformers4Rec version: 23.12.0
  • Platform: Ubuntu 20.04.3 LTS
  • Python version: 3.8.10
  • Huggingface Transformers version: 4.30.2
  • PyTorch version (GPU?): 2.4.1
  • Tensorflow version (GPU?): 2.7.0
Additional context

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in data_utils.py at save_time_based_splits and _save_time_based_splits_cpu, then reproduce the issue using the CPU=True example from the linked notebook. Done means the function raises no exception and splits the data as expected.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.