NVIDIA-Merlin / NVIDIA-Merlin/Transformers4Rec
[BUG] save_time_based_splits function does not support CPU mode well
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.3k
- Forks
- 165
- Avg merge
- 1m
- Merged PRs (30d)
- 2
Description
Bug description
Function save_time_based_splits in data_utils.py does not support CPU mode correctly. In particular, function _save_time_based_splits_cpu assumes using Rapids libraries, moreover Dask Dataframe seems incorrectly imported.
Steps/Code to reproduce bug
Using code from examples, just with option CPU set to True (https://github.com/NVIDIA-Merlin/Transformers4Rec/blob/main/examples/getting-started-session-based/01-ETL-with-NVTabular.ipynb)
sessions_gdf = df.read_parquet(BASE_PATH / "processed_nvt/part_0.parquet")
from transformers4rec.utils.data_utils import save_time_based_splits
save_time_based_splits(
data=nvt.Dataset(sessions_gdf),
output_dir=BASE_PATH / f"session_by_day",
partition_col="day-first",
timestamp_col="session_id",
cpu=True
)
Expected behavior
No exception is thrown and data are splitted.
Environment details
- Transformers4Rec version: 23.12.0
- Platform: Ubuntu 20.04.3 LTS
- Python version: 3.8.10
- Huggingface Transformers version: 4.30.2
- PyTorch version (GPU?): 2.4.1
- Tensorflow version (GPU?): 2.7.0
Additional context
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in data_utils.py at save_time_based_splits and _save_time_based_splits_cpu, then reproduce the issue using the CPU=True example from the linked notebook. Done means the function raises no exception and splits the data as expected.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100