NVIDIA-Merlin / NVIDIA-Merlin/Transformers4Rec

[Task] Use the same logic of the new merlin DataLoader to set the features types in T4Rec

Open
#524 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
1.3k
Forks
165
Avg merge
1m
Merged PRs (30d)
2

Description

Description

  • The previous nvtabular data loader was always returning variables with the dtype float32 and int32. We followed the same standard format in T4Rec which requires inputs to be of float32/int32.

  • The new data loader changed the convention of always returning float32/int32 to return the same type used in the source data being loaded (e.g. from a parquet file).

  • This change in data loader causes an error in T4rec when the parquet file was containing features with float64 dtypes ( more details here).

  • This feature request aims to extend T4Rec to use the same types used in the source data being loaded

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing T4Rec's handling of feature dtypes when data comes from the new Merlin DataLoader, using the float64 parquet-file failure described in the issue and the linked NVTabular issue for context. Done means T4Rec accepts and uses the source feature types rather than requiring only float32 and int32 inputs.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
data, machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.