marshmallow-code / marshmallow-code/marshmallow

Schema fields with dict attribute names return dict attributes when keys missing

Open
#648 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feedback welcome needs review
Dominant language
Python
Stars
7.2k
Forks
738
Avg merge
1d 23h
Merged PRs (30d)
7

Description

Whenever a Schema is defined that has fields that shadow dict attributes (e.g. "items", "keys", "values", etc), passing a dict with those keys not present to Schema.dump() results in dict attributes being serialized:

from marshmallow import Schema, fields

class A(Schema):
    a = fields.Str()
    keys = fields.Str(default='k')
    values = fields.Str(missing='v')
    items = fields.Str()

# No problem on load.
loaded = A().load({'a': 'x'})
print(loaded.data)
# {'a': 'x', 'values': 'v'}

# No problem when fields present.
dumped = A().dump({'a': 'x', 'keys': 'k', 'values': 'v', 'items': 'i'})
print(dumped.data)
# {'a': 'x', 'keys': 'k', 'items': 'i', 'values': 'v'}

# Undesired attribute access when fields not present.
dumped = A().dump({'a': 'x'})
print(dumped.data)

# on marshmallow==2.13.5
# {'a': 'x',
#  'keys': "dict_keys(['a'])",
#  'values': "dict_values(['x'])",
#  'items': "dict_items([('a', 'x')])"}

# on marshmallow==3.0.0b2
# {'a': 'x',
#  'keys': '<built-in method keys of dict object at 0x7f4e206bc948>'}
#  'values': '<built-in method values of dict object at 0x7f4e206bc948>',
#  'items': '<built-in method items of dict object at 0x7f4e206bc948>',

I found #30 which indicates that this is intended behavior for dumping dict objects, but I was wondering if 3.0.0 may be open to changing it (for dict classes only) or providing an option to disable dict-attribute access when dumping.

I'm aware that I could change the attribute name in the Schema definition and utilize load_from/dump_to but I wanted to avoid aliasing to keep the Schema attributes the same as the de-/serialization format.

In my own application, I've overridden Schema.get_attribute with a custom implementation so that dict attributes aren't accessed at all but it would be nice to not to have to do this in each project.

If overriding Schema.get_attribute is the preferred method for implementing this functionality (instead of having a first-class option/setting), then I'll keep doing that. However, it may be worth adding a warning or note in the documentation about this "gotcha" when dumping a dict object with a Schema that mirrors dict attributes. The section on Specifying Attribute Names does mention object attribute access but might be helpful to call out the potential field/dict-attribute conflicts that can result when dumping (unless I'm missing something else in the docs that covers this scenario).

Thoughts?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing Schema.get_attribute and the documentation section on Specifying Attribute Names referenced in the issue. Compare the dict-attribute behavior for missing fields with the intended marshmallow 3 behavior, then identify whether the resolved scope is a code change, an option, or documentation; done requires an agreed behavior and corresponding coverage or documentation.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.