NVIDIA-Merlin / NVIDIA-Merlin/Transformers4Rec
[SECURITY] Pickle allowlist in `utils/serialization.py` is bypassable (`builtins.getattr` allowed, duplicate keys drop `PredictionTask`)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.3k
- Forks
- 165
- Avg merge
- 1m
- Merged PRs (30d)
- 2
Description
Bug description
The BASE_SERIALIZATION_CLASSES allowlist introduced by PR #802 (commit ab7207cf, "Sec pic fix") in transformers4rec/utils/serialization.py has two structural issues that defeat the intended restriction on Unpickler.find_class.
File (at ab7207cf):
https://github.com/NVIDIA-Merlin/Transformers4Rec/blob/ab7207cf40c7960f7aa22c86ab232576aa8cf847/transformers4rec/utils/serialization.py#L11-L68
The file was subsequently removed in PR #808 (41b14d7b), but checkpoints produced during the #802 → #807 window still exist in the wild and the design questions apply to any follow-up allowlist.
(1) builtins key appears twice. Python dict literals are last-write-wins, so only the second declaration survives:
BASE_SERIALIZATION_CLASSES = {
"builtins": [
"Exception", "ValueError", "NotImplementedError", "AttributeError",
"AssertionError"
],
...
"builtins": ["getattr"], # <-- this overrides the first "builtins" key
}
Net result at runtime: builtins maps to ["getattr"]. The Exception subclasses (intentionally allowlisted) are silently dropped, and builtins.getattr — a well-known primitive in pickle gadget chains (attribute traversal → code execution) — is approved.
(2) torch.storage._load_from_bytes is in the allowlist. _load_from_bytes wraps torch.load, which itself performs unrestricted pickle deserialization when the installed PyTorch version predates the weights_only=True default. Combined with getattr, this provides a reachable path from the restricted unpickler to arbitrary code execution.
(3) transformers4rec.torch.model.base is also duplicated. The first declaration includes PredictionTask; the second (which wins) does not:
"transformers4rec.torch.model.base": ["Model", "Head", "PredictionTask"],
...
"transformers4rec.torch.model.base": ["forward_to_prediction_fn", "Model", "Head"], # wins; PredictionTask dropped
So any checkpoint containing PredictionTask fails to deserialize with ValueError from the restricted unpickler. This is an availability regression, not a security one, but it indicates the dict literal was not reviewed carefully before merge.
Steps/Code to reproduce bug
Structural evidence (no exploit payload):
git show ab7207cf:transformers4rec/utils/serialization.py | grep -n '"builtins"'
# 12: "builtins": [
# 66: "builtins": ["getattr"],
git show ab7207cf:transformers4rec/utils/serialization.py | grep -n 'torch.storage'
# 49: "torch.storage": ["_load_from_bytes"],
Evaluating the module confirms the surviving values:
import ast
src = open(".../serialization.py").read()
ns = {}
exec(src, ns)
print(ns["BASE_SERIALIZATION_CLASSES"]["builtins"]) # ['getattr']
print(ns["BASE_SERIALIZATION_CLASSES"]["torch.storage"]) # ['_load_from_bytes']
print("PredictionTask" in ns["BASE_SERIALIZATION_CLASSES"]
["transformers4rec.torch.model.base"]) # False
I am intentionally not attaching an end-to-end exploit payload to a public issue; the combination of builtins.getattr + torch.storage._load_from_bytes is sufficient for any reader familiar with pickle gadget chains to reproduce.
Expected behavior
- The allowlist does not contain
builtins.getattrortorch.storage._load_from_bytes. - The dict has no duplicate keys (a
flake8-bugbear B033or similar lint would have caught this). - The design shifts away from pickle-based serialization for model weights: either
safetensors, ortorch.load(..., weights_only=True)after bumping the minimum PyTorch version (see the companiontorch.loadissue I am filing).
Environment details
- Transformers4Rec: commits
ab7207cf(PR #802) through0e31f575(PR #807); file removed at41b14d7b(PR #808) but affected checkpoints persist. - Python: any
- PyTorch: any (the
_load_from_bytesgadget exists across all modern versions)
Additional context
Suggested remediation order:
- Restore allowlist-based protection only as a temporary measure — remove
builtins.getattrandtorch.storage._load_from_bytes, de-duplicate all keys. - Transition model-weight serialization to a format that does not execute code on load (
safetensorsis the common choice for HF-adjacent projects). - For any remaining pickle-based artifacts, wrap
torch.loadwith an explicitweights_only=True(see the companion issue).
I can prepare separate PRs for each step if the maintainers prefer.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by inspecting the historical transformers4rec/utils/serialization.py from commit ab7207cf and compare it with the current serialization paths, since the file was removed in PR #808. Verify the duplicate-key behavior and the allowed torch.storage entry, then review the companion torch.load issue. Done means no unsafe entries or duplicate keys remain and a safe replacement path is defined for affected checkpoints.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- security
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100