NVIDIA-Merlin / NVIDIA-Merlin/Transformers4Rec

[BUG] `cannot pickle 'mappingproxy' object` when using `TabularFeatures` `create_categorical`

Open
#727 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug status/needs-triage
Dominant language
Python
Stars
1.3k
Forks
165
Avg merge
1m
Merged PRs (30d)
2

Description

Bug description

I have a bug just creating a schema programmatically. Can you help me on this?

thx

Steps/Code to reproduce bug
import merlin_standard_lib as msl
from merlin_standard_lib import Schema
from transformers4rec.torch.features.tabular import TabularFeatures


features_schema = Schema([msl.ColumnSchema.create_categorical("language", num_items=149),]
        )
a = TabularFeatures.from_schema(
        features_schema,
    )

I have TypeError: cannot pickle 'mappingproxy' object

coming from

│ /home/mdenadai/miniconda3/envs/gnn/lib/python3.9/site-packages/transformers4rec/torch/features/t │
│ abular.py:175 in from_schema                                                                     │
│                                                                                                  │
│   172 │   │   │   │   │   **kwargs,                                                              │
│   173 │   │   │   │   )                                                                          │
│   174 │   │   │   else:                                                                          │
│ ❱ 175 │   │   │   │   maybe_continuous_module = cls.CONTINUOUS_MODULE_CLASS.from_schema(         │
│   176 │   │   │   │   │   schema, tags=continuous_tags, **kwargs                                 │
│   177 │   │   │   │   )                                                                          │
│   178 │   │   if categorical_tags:                                                               │
│                                                                                                  │
│ /home/mdenadai/miniconda3/envs/gnn/lib/python3.9/site-packages/transformers4rec/torch/tabular/ba │
│ se.py:190 in from_schema                                                                         │
│                                                                                                  │
│   187 │   │   -------                                                                            │
│   188 │   │   Optional[TabularModule]                                                            │
│   189 │   │   """                                                                                │
│ ❱ 190 │   │   schema_copy = deepcopy(schema)                                                     │
│   191 │   │   if tags:                                                                           │
│   192 │   │   │   schema_copy = schema_copy.select_by_tag(tags)

This happens even when I just do:

import deepcopy
import merlin_standard_lib as msl
from merlin_standard_lib import Schema
from transformers4rec.torch.features.tabular import TabularFeatures

deepcopy(Schema([msl.ColumnSchema.create_categorical("language", num_items=149),]))

Environment details
  • Transformers4Rec version: 23.6.0
  • Platform: unix
  • Python version: 3.9

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the failure with the shown Schema and deepcopy call, then inspect transformers4rec/torch/features/tabular.py and transformers4rec/torch/tabular/base.py around TabularFeatures.from_schema and the deepcopy at line 190. Confirm that deepcopy succeeds for the categorical schema and that TabularFeatures.from_schema completes without the mappingproxy TypeError.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.