NVIDIA-Merlin / NVIDIA-Merlin/Transformers4Rec

[BUG] `MerlinDataLoader._augment_schema` raises TypeError when `lists` is omitted

Open Beginner friendly
#812 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.3k
Forks
165
Avg merge
1m
Merged PRs (30d)
2

Description

Bug description

transformers4rec/torch/utils/data_utils.py normalizes cats, conts, and labels to empty lists when they default to None, but does the same for lists:

https://github.com/NVIDIA-Merlin/Transformers4Rec/blob/8bf122f5dcb39feecfc6dabde734d79c2d1c4380/transformers4rec/torch/utils/data_utils.py#L390-L412

@staticmethod
def _augment_schema(
    schema,
    cats=None,
    conts=None,
    labels=None,
    lists=None,
):
    cats = cats or []
    conts = conts or []
    labels = labels or []
    # <-- missing: lists = lists or []

    schema = schema.select_by_name(conts + cats + labels + lists)
    ...
    for col in lists:
        schema[col] = schema[col].with_tags(Tags.LIST)

The expression conts + cats + labels + lists raises TypeError: can only concatenate list (not "NoneType") to list for any caller that omits lists= (i.e. does not have list-typed features). The for col in lists loop later would also raise TypeError: 'NoneType' object is not iterable.

Since list-typed features are optional in the dataloader, omitting lists= is the common case.

Steps/Code to reproduce bug

Extracted repro (no T4Rec install needed):

class Schema:
    def select_by_name(self, cols):
        return self

def _augment_schema(schema, cats=None, conts=None, labels=None, lists=None):
    cats = cats or []
    conts = conts or []
    labels = labels or []
    return schema.select_by_name(conts + cats + labels + lists)

_augment_schema(Schema(), cats=["x"])
# TypeError: can only concatenate list (not "NoneType") to list
Expected behavior

Omitting lists= should behave identically to lists=[].

Environment details
  • Transformers4Rec: main @ 8bf122f5
  • Python: any
Additional context

One-line fix:

cats = cats or []
conts = conts or []
labels = labels or []
lists = lists or []

This may explain user reports like #797 where pre-trained-embedding pipelines fall over under real datasets — _augment_schema is on the construction path of MerlinDataLoader for any non-list dataset. Happy to send a PR.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Open transformers4rec/torch/utils/data_utils.py around MerlinDataLoader._augment_schema at lines 390-412, then run the extracted reproduction from the issue or the relevant dataloader tests. Verify that omitting lists follows the same path as lists=[] and that schema selection and list-column handling no longer raise TypeError.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
data
Issue type
Bug
Difficulty
1/5
Estimated time
Under an hour
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
88/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.