marshmallow-code / marshmallow-code/marshmallow

Different order of pre_load hooks for List(Nested) and Nested(many=True) fields

Open
#1,942 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
7.2k
Forks
738
Avg merge
1d 23h
Merged PRs (30d)
7

Description

Hello, I found a strange difference in load behavior between List(Nested) and Nested(many=True)
Consider the following example:
```python
from copy import deepcopy
from marshmallow import Schema, fields, pre_load

class InnerSchema(Schema):
value = fields.String()

@pre_load(pass_many=False)
def add_prefix(self, data, **_):
print('pre_load inner')
data = deepcopy(data)
data['value'] = '_'.join([self.context.get('prefix'), data['value']])
return data

class MiddleSchema(Schema):
prefix = fields.String()
inner = fields.Nested(InnerSchema)

@pre_load(pass_many=False)
def store_prefix(self, data, **_):
print('pre_load middle')
self.context['prefix'] = data['prefix']
return data

class ListNestedSchema(Schema):
data = fields.List(fields.Nested(MiddleSchema))

class NestedManySchema(Schema):
data = fields.Nested(MiddleSchema, many=True)

obj = {'data': [
{'prefix': 'foo', 'inner': {'value': 'x'}},
{'prefix': 'bar', 'inner': {'value': 'z'}}
]}

```
In this synthetic example I have a list of objects, and I need each of them to pass its own piece if data to nested ones on loading.
I'm using `context` to pass this piece of data to nested schema and expecting each of `InnerSchema` to receive its own piece of data corresponding to exact parent object.
Loading data with `ListNestedSchema` does exactly what I expect, however `NestedManySchema` works in a different manner. It looks like the reason is that the order of `pre_load` hooks execution differs and I'm not sure if it was designed in such way or is it a bug.

```python
>>> ListNestedSchema().load(obj)
pre_load middle
pre_load inner
pre_load middle
pre_load inner
{'data': [{'inner': {'value': 'foo_x'}, 'prefix': 'foo'}, {'inner': {'value': 'bar_z'}, 'prefix': 'bar'}]}

>>> NestedManySchema().load(obj)
pre_load middle
pre_load middle
pre_load inner
pre_load inner
{'data': [{'inner': {'value': 'bar_x'}, 'prefix': 'foo'}, {'inner': {'value': 'bar_z'}, 'prefix': 'bar'}]}
```

I'm using marshmallow==3.14.1 with Python 3.8.2

Contributor guide

Open the contributing guide

Research direction

Reproduce the difference with the ListNestedSchema and NestedManySchema examples in the issue, then trace how pre_load hooks are invoked for fields.List(fields.Nested(...)) versus fields.Nested(..., many=True). Done means the behavior is made consistent or explicitly documented, with regression coverage for per-item context propagation.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.