python-attrs / python-attrs/attrs

Mixed inheritance between slotted and non slotted classes leads to issues with parallelisation.

Open
#1,020 14 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Documentation Thinking
Dominant language
Python
Stars
5.8k
Forks
480
Avg merge
2h 15m
Merged PRs (30d)
2

Description

Mixed inheritance between slotted and non-slotted classes seems to lead to some issues when object are copied for parallelization.

The following code example does not work and raises: AttributeError: 'Child' object has no attribute 'z'

import attrs
import multiprocessing as mp

@attrs.define(slots=False, kw_only=True)
class Parent:
    name: str 
    
@attrs.define
class Child(Parent):
    x: float
    
    def __attrs_post_init__(self):
        self.z = self.x + 2

    def arg_plus_z(self, arg):
        return self.z + arg

test = Child(3, name = "as")

with mp.Pool(processes=4) as pool:
    result = pool.map(test.arg_plus_z, range(100))

whilst adding slots=False to the child class or registering z using z: float = attrs.field(default = None, init=False) does work:

import attrs
import multiprocessing as mp

@attrs.define(slots=False, kw_only=True)
class Parent:
    name: str 
    
@attrs.define(slots=False)
class Child(Parent):
    x: float
    
    def __attrs_post_init__(self):
        self.z = self.x + 2

    def arg_plus_z(self, arg):
        return self.z + arg

test = Child(3, name = "as")

with mp.Pool(processes=4) as pool:
    result = pool.map(test.arg_plus_z, range(100))

It seems like the object copy of test passed to each one of the processes does not copy the __dict__ (which includes z and name) but only the __slots__.

I know that according to the attrs documentation mixed inheritance between slotted and non slotted classes is considered a bad practice. I stumbled across this by accident and it might also be more of a missing feature than a bug.

Meta:

  • attrs-version: '21.4.0'
  • python-version: '3.9.13'

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the provided mixed-inheritance reproducer and inspect how the Child instance is copied for multiprocessing, particularly the interaction between dict and slots. The work is done when the copied object retains z and name and the pool example succeeds, with regression coverage for the failing case.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.