marshmallow-code / marshmallow-code/marshmallow

Accessing builtin dict's attrs for a missing field

Open
#1,559 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
7.2k
Forks
738
Avg merge
1d 23h
Merged PRs (30d)
7

Description

Referring the PR #1557

I've faced this issue, when a schema has a field with name same as builtin dict's has attribute:

```python
from marshmallow import fields, Schema

class MyNestedSchema(Schema):
update = fields.Boolean(required=False)

class MySchema(Schema):
data = fields.Nested(MyNestedSchema)

class MyObj:
def __init__(self):
self.data = {
# NOT passing a value, so it's missing
# 'update': True,
}

def main():
obj = MyObj()
data = MySchema().dump(obj)
print(data)

if __name__ == '__main__':
main()
```

Fails with this:

```
Traceback (most recent call last):
File "/home/khorenyan/.PyCharm2019.2/config/scratches/scratch_42.py", line 40, in
main()
File "/home/khorenyan/.PyCharm2019.2/config/scratches/scratch_42.py", line 35, in main
data = MySchema().dump(obj)
File "/home/khorenyan/projs/marshmallow/src/marshmallow/schema.py", line 556, in dump
result = self._serialize(processed_obj, many=many)
File "/home/khorenyan/projs/marshmallow/src/marshmallow/schema.py", line 520, in _serialize
value = field_obj.serialize(attr_name, obj, accessor=self.get_attribute)
File "/home/khorenyan/projs/marshmallow/src/marshmallow/fields.py", line 311, in serialize
return self._serialize(value, attr, obj, **kwargs)
File "/home/khorenyan/projs/marshmallow/src/marshmallow/fields.py", line 566, in _serialize
return schema.dump(nested_obj, many=many)
File "/home/khorenyan/projs/marshmallow/src/marshmallow/schema.py", line 556, in dump
result = self._serialize(processed_obj, many=many)
File "/home/khorenyan/projs/marshmallow/src/marshmallow/schema.py", line 520, in _serialize
value = field_obj.serialize(attr_name, obj, accessor=self.get_attribute)
File "/home/khorenyan/projs/marshmallow/src/marshmallow/fields.py", line 311, in serialize
return self._serialize(value, attr, obj, **kwargs)
File "/home/khorenyan/projs/marshmallow/src/marshmallow/fields.py", line 1103, in _serialize
elif value in self.truthy:
TypeError: unhashable type: 'dict'
```

It happens this way:
- When being serialized, object is passed to the `marshmallow.utils._get_value_for_key`
- because of the missing `update` field in data on the line `obj[key]` the `KeyError` exception is raised
- and on the next line the `getattr(obj, key, default)` is returned

If the `key` is not found here, it returns default, so if my field is `'update_'` or `'qwerty'`, it works as expected -- the result is returned and it's `{'data': {}}`

But the `'update'` matches the builtin dict's attribute (method) so it's returned. And that's not expected at all.

Of course, I could override the `get_attribute` method in `MyNestedSchema` this way:

```python
class MyNestedSchema(Schema):
def get_attribute(self, obj: dict, attr: str, default: Any):
try:
return obj[attr]
except KeyError:
return default

update = fields.Boolean(required=False)
```

But I think, that needs to be fixed.

I thought about checking if the `getattr(obj, key, default)` expression returns a method of a builtin's object, or checking a key to be one of the builtin's attrs, for example:

```python
>>> dir(dict)
['__class__', '__contains__', '__delattr__', '__delitem__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__iter__', '__le__', '__len__', '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setitem__', '__sizeof__', '__str__', '__subclasshook__', 'clear', 'copy', 'fromkeys', 'get', 'items', 'keys', 'pop', 'popitem', 'setdefault', 'update', 'values']
```

I'm not sure if that's a proper way to do this check. Maybe we need to check even for `isinstance` instead of the object's type, but here's my current solution.

@deckar01 suggested discussing implementing something like `schema.dump(obj, from_obj=dict)`

Contributor guide

Open the contributing guide

Research direction

Start in marshmallow.utils._get_value_for_key and follow its use from src/marshmallow/schema.py and src/marshmallow/fields.py. Reproduce the nested-schema example with a missing update field, then verify that dictionary keys shadowing dict attributes are treated as missing rather than serialized as methods, without regressing ordinary attribute access.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.