marshmallow-code / marshmallow-code/marshmallow
Accessing builtin dict's attrs for a missing field
- Dominant language
- Python
- Stars
- 7.2k
- Forks
- 738
- Avg merge
- 1d 23h
- Merged PRs (30d)
- 7
Description
Referring the PR #1557
I've faced this issue, when a schema has a field with name same as builtin dict's has attribute:
```python
from marshmallow import fields, Schema
class MyNestedSchema(Schema):
update = fields.Boolean(required=False)
class MySchema(Schema):
data = fields.Nested(MyNestedSchema)
class MyObj:
def __init__(self):
self.data = {
# NOT passing a value, so it's missing
# 'update': True,
}
def main():
obj = MyObj()
data = MySchema().dump(obj)
print(data)
if __name__ == '__main__':
main()
```
Fails with this:
```
Traceback (most recent call last):
File "/home/khorenyan/.PyCharm2019.2/config/scratches/scratch_42.py", line 40, in
main()
File "/home/khorenyan/.PyCharm2019.2/config/scratches/scratch_42.py", line 35, in main
data = MySchema().dump(obj)
File "/home/khorenyan/projs/marshmallow/src/marshmallow/schema.py", line 556, in dump
result = self._serialize(processed_obj, many=many)
File "/home/khorenyan/projs/marshmallow/src/marshmallow/schema.py", line 520, in _serialize
value = field_obj.serialize(attr_name, obj, accessor=self.get_attribute)
File "/home/khorenyan/projs/marshmallow/src/marshmallow/fields.py", line 311, in serialize
return self._serialize(value, attr, obj, **kwargs)
File "/home/khorenyan/projs/marshmallow/src/marshmallow/fields.py", line 566, in _serialize
return schema.dump(nested_obj, many=many)
File "/home/khorenyan/projs/marshmallow/src/marshmallow/schema.py", line 556, in dump
result = self._serialize(processed_obj, many=many)
File "/home/khorenyan/projs/marshmallow/src/marshmallow/schema.py", line 520, in _serialize
value = field_obj.serialize(attr_name, obj, accessor=self.get_attribute)
File "/home/khorenyan/projs/marshmallow/src/marshmallow/fields.py", line 311, in serialize
return self._serialize(value, attr, obj, **kwargs)
File "/home/khorenyan/projs/marshmallow/src/marshmallow/fields.py", line 1103, in _serialize
elif value in self.truthy:
TypeError: unhashable type: 'dict'
```
It happens this way:
- When being serialized, object is passed to the `marshmallow.utils._get_value_for_key`
- because of the missing `update` field in data on the line `obj[key]` the `KeyError` exception is raised
- and on the next line the `getattr(obj, key, default)` is returned
If the `key` is not found here, it returns default, so if my field is `'update_'` or `'qwerty'`, it works as expected -- the result is returned and it's `{'data': {}}`
But the `'update'` matches the builtin dict's attribute (method) so it's returned. And that's not expected at all.
Of course, I could override the `get_attribute` method in `MyNestedSchema` this way:
```python
class MyNestedSchema(Schema):
def get_attribute(self, obj: dict, attr: str, default: Any):
try:
return obj[attr]
except KeyError:
return default
update = fields.Boolean(required=False)
```
But I think, that needs to be fixed.
I thought about checking if the `getattr(obj, key, default)` expression returns a method of a builtin's object, or checking a key to be one of the builtin's attrs, for example:
```python
>>> dir(dict)
['__class__', '__contains__', '__delattr__', '__delitem__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__iter__', '__le__', '__len__', '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setitem__', '__sizeof__', '__str__', '__subclasshook__', 'clear', 'copy', 'fromkeys', 'get', 'items', 'keys', 'pop', 'popitem', 'setdefault', 'update', 'values']
```
I'm not sure if that's a proper way to do this check. Maybe we need to check even for `isinstance` instead of the object's type, but here's my current solution.
@deckar01 suggested discussing implementing something like `schema.dump(obj, from_obj=dict)`
Contributor guide
Research direction
Start in marshmallow.utils._get_value_for_key and follow its use from src/marshmallow/schema.py and src/marshmallow/fields.py. Reproduce the nested-schema example with a missing update field, then verify that dictionary keys shadowing dict attributes are treated as missing rather than serialized as methods, without regressing ordinary attribute access.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100