python / python/cpython

Objects with both `__slots__` and `__dict__` have much larger size than needed (up to 4x) on Python 3.13

Open
#135,385 4 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

3.13 3.14 interpreter-core performance type-bug
Dominant language
Python
Stars
77.2k
Forks
36k
PR merge metrics
PR metrics pending

Description

Bug report

Bug description:

On Python 3.13, instances that have both __slots__ and __dict__ defined (e.g. through inheritance)
can have a size that is more than four times of that in previous Python versions (e.g. 3.12).

import tracemalloc
import gc


class _Point2D:
    __slots__ = ("x", "y")


class Point3D_OnlySlots(_Point2D):
    __slots__ = ("z",)

    def __init__(self, x, y, z):
        self.x, self.y, self.z = x, y, z


class Point3D_DictAndSlots(_Point2D):
    def __init__(self, x, y, z):
        self.x, self.y, self.z = x, y, z


class Point3D_OnlyDict:
    def __init__(self, x, y, z):
        self.x, self.y, self.z = x, y, z


gc.collect()  # clear freelists
tracemalloc.start()
_ = [Point3D_OnlySlots(1, 2, 3) for _ in range(1_000_000)]
print(
    f"1M Point3D (only __slots__) instances:         {tracemalloc.get_traced_memory()[0]:_} bytes"
)

gc.collect()  # clear freelists
tracemalloc.start()
_ = [Point3D_OnlyDict(1, 2, 3) for _ in range(1_000_000)]
print(
    f"1M Point3D (only __dict__) instances:          {tracemalloc.get_traced_memory()[0]:_} bytes"
)

gc.collect()  # clear freelists
tracemalloc.start()
_ = [Point3D_DictAndSlots(1, 2, 3) for _ in range(1_000_000)]
print(
    f"1M Point3D (__dict__ and __slots__) instances: {tracemalloc.get_traced_memory()[0]:_} bytes"
)

On python 3.13.3, this prints:

1M Point3D (only __slots__) instances:         64_448_792 bytes
1M Point3D (only __dict__) instances:          104_451_928 bytes
1M Point3D (__dict__ and __slots__) instances: 416_448_848 bytes

On python 3.12.11, this prints:

1M Point3D (only __slots__) instances:         64_448_792 bytes
1M Point3D (only __dict__) instances:          96_451_808 bytes
1M Point3D (__dict__ and __slots__) instances: 96_452_232 bytes

The apparent cause seems to be the object layout changes (see here)
which introduced inline values. It appears that these don't mesh well with __slots__.
Could this be because the inline values need to have a fixed offset?
What appears to cause the dramatic increase above is that having (nonempty) __slots__ will
trigger __dict__ materialization (size 30) as soon as a non-slot attribute is set. In contrast to the optimized "no slots" case, this dict doesn't shrink as more instances are created.

Of course, mixing __slots__ and __dict__ is not recommended, but it's a trap that can be easily fallen into.
For example, if forgetting to define __slots__ anywhere in a complex class hierarchy.

I'm unsure if I'm understanding exactly what's going on though. @markshannon perhaps you can shed some light on this?

Related: #115776, #115822

CPython versions tested on:

3.13

Operating systems tested on:

macOS

Linked PRs
  • gh-135389
  • gh-156656

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the object layout changes described in Objects/object_layout.md and reproduce the issue using the supplied slots/dict classes with tracemalloc on Python 3.13 and 3.12. Review linked PRs gh-135389 and gh-156656, then verify that mixed slots-and-dict instances no longer incur the reported excess memory while preserving the demonstrated layouts.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.