NVIDIA-Merlin / NVIDIA-Merlin/Transformers4Rec
[BUG] `MerlinDataLoader._augment_schema` raises TypeError when `lists` is omitted
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.3k
- Forks
- 165
- Avg merge
- 1m
- Merged PRs (30d)
- 2
Description
Bug description
transformers4rec/torch/utils/data_utils.py normalizes cats, conts, and labels to empty lists when they default to None, but does the same for lists:
@staticmethod
def _augment_schema(
schema,
cats=None,
conts=None,
labels=None,
lists=None,
):
cats = cats or []
conts = conts or []
labels = labels or []
# <-- missing: lists = lists or []
schema = schema.select_by_name(conts + cats + labels + lists)
...
for col in lists:
schema[col] = schema[col].with_tags(Tags.LIST)
The expression conts + cats + labels + lists raises TypeError: can only concatenate list (not "NoneType") to list for any caller that omits lists= (i.e. does not have list-typed features). The for col in lists loop later would also raise TypeError: 'NoneType' object is not iterable.
Since list-typed features are optional in the dataloader, omitting lists= is the common case.
Steps/Code to reproduce bug
Extracted repro (no T4Rec install needed):
class Schema:
def select_by_name(self, cols):
return self
def _augment_schema(schema, cats=None, conts=None, labels=None, lists=None):
cats = cats or []
conts = conts or []
labels = labels or []
return schema.select_by_name(conts + cats + labels + lists)
_augment_schema(Schema(), cats=["x"])
# TypeError: can only concatenate list (not "NoneType") to list
Expected behavior
Omitting lists= should behave identically to lists=[].
Environment details
- Transformers4Rec:
main@8bf122f5 - Python: any
Additional context
One-line fix:
cats = cats or []
conts = conts or []
labels = labels or []
lists = lists or []
This may explain user reports like #797 where pre-trained-embedding pipelines fall over under real datasets — _augment_schema is on the construction path of MerlinDataLoader for any non-list dataset. Happy to send a PR.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Open transformers4rec/torch/utils/data_utils.py around MerlinDataLoader._augment_schema at lines 390-412, then run the extracted reproduction from the issue or the relevant dataloader tests. Verify that omitting lists follows the same path as lists=[] and that schema selection and list-column handling no longer raise TypeError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- data
- Issue type
- Bug
- Difficulty
- 1/5
- Estimated time
- Under an hour
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 88/100