python-pillow / python-pillow/Pillow

Memory of copied PIL Images is not released

Open
#7,935 13 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Memory
Dominant language
Python
Stars
13.8k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
89

Description

What did you do?

Our application works with PIL images and holds a list of containers. Every container has a copy of the last image to track manipulations of the image data. When we delete the containers, the memory reserved by the PIL images is not released. Even closing the image manually via image.close() in the container's desctructor and calling the garbage collector does not release the memory.

If I replace the PIL image with a Python list (line: 55) the memory gets freed when a container is popped from the list.

import gc
import os
import sys
import time

import PIL.Image
import psutil

FILE = './test_image.png'


def LogMemory():
    pid = os.getpid()
    rss = 0
    for mmap in psutil.Process(pid).memory_maps():
        # All memory that this process holds in RAM. RSS = USS + Shared.
        rss += mmap.rss
    print(f'RSS: {rss}')


class Container:

    def __init__(self):
        self.value = None

    def __del__(self):
        if isinstance(self.value, PIL.Image.Image):
            LogMemory()
            self.value.close()
            self.value = None
            gc.collect()
            print('closed image in destructor')
            LogMemory()

    def SetValue(self, value):
        self.value = self._copyValue(value)

    def GetValue(self):
        return self._copyValue(self.value)

    def _copyValue(self, value):
        return value.copy()


if __name__ == '__main__':

    print(f'Using Python {sys.version}, PIL {PIL.__version__}')

    containers = []

    for i in range(50):
        print(f'Load image {i}')
        LogMemory()
        img = PIL.Image.open(FILE)
        # img = [1] * (1920*1080*3)  # this works!
        container = Container()
        container.SetValue(img)
        containers.append(container)
        LogMemory()

    for i in range(len(containers)):
        print(f'pop container {i}')
        LogMemory()
        containers.pop()
        time.sleep(0.1)
        LogMemory()

    print('Delete list')
    LogMemory()
    containers = None
    gc.collect()
    LogMemory()

test_image

What did you expect to happen?

The memory, taken by a PIL image copy, should be released after each containers.pop().

What actually happened?

The memory isn't released.

Script output 
```text
Using Python 3.11.8 (main, Feb 25 2024, 16:41:26) [GCC 9.4.0], PIL 10.3.0
Load image 0
RSS: 21696512
RSS: 40095744
Load image 1
RSS: 40108032
RSS: 48123904
Load image 2
RSS: 48123904
RSS: 56512512
Load image 3
RSS: 56512512
RSS: 64679936
Load image 4
RSS: 64679936
RSS: 72974336
Load image 5
RSS: 72974336
RSS: 81268736
Load image 6
RSS: 81268736
RSS: 89563136
Load image 7
RSS: 89563136
RSS: 97857536
Load image 8
RSS: 97857536
RSS: 106151936
Load image 9
RSS: 106151936
RSS: 114446336
Load image 10
RSS: 114446336
RSS: 122740736
Load image 11
RSS: 122740736
RSS: 131035136
Load image 12
RSS: 131035136
RSS: 139329536
Load image 13
RSS: 139329536
RSS: 147623936
Load image 14
RSS: 147623936
RSS: 155918336
Load image 15
RSS: 155918336
RSS: 164212736
Load image 16
RSS: 164212736
RSS: 172507136
Load image 17
RSS: 172507136
RSS: 180801536
Load image 18
RSS: 180801536
RSS: 189095936
Load image 19
RSS: 189095936
RSS: 197390336
Load image 20
RSS: 197390336
RSS: 205684736
Load image 21
RSS: 205684736
RSS: 213979136
Load image 22
RSS: 213979136
RSS: 222273536
Load image 23
RSS: 222273536
RSS: 230576128
Load image 24
RSS: 230576128
RSS: 238964736
Load image 25
RSS: 238964736
RSS: 247156736
Load image 26
RSS: 247156736
RSS: 255451136
Load image 27
RSS: 255451136
RSS: 263745536
Load image 28
RSS: 263745536
RSS: 272039936
Load image 29
RSS: 272039936
RSS: 280334336
Load image 30
RSS: 280334336
RSS: 288628736
Load image 31
RSS: 288628736
RSS: 296923136
Load image 32
RSS: 296923136
RSS: 305217536
Load image 33
RSS: 305217536
RSS: 313511936
Load image 34
RSS: 313511936
RSS: 321806336
Load image 35
RSS: 321806336
RSS: 330100736
Load image 36
RSS: 330100736
RSS: 338395136
Load image 37
RSS: 338395136
RSS: 346689536
Load image 38
RSS: 346689536
RSS: 354983936
Load image 39
RSS: 354983936
RSS: 363278336
Load image 40
RSS: 363278336
RSS: 371572736
Load image 41
RSS: 371572736
RSS: 379867136
Load image 42
RSS: 379867136
RSS: 388161536
Load image 43
RSS: 388161536
RSS: 396455936
Load image 44
RSS: 396455936
RSS: 404750336
Load image 45
RSS: 404750336
RSS: 413044736
Load image 46
RSS: 413044736
RSS: 421416960
Load image 47
RSS: 421416960
RSS: 429633536
Load image 48
RSS: 429633536
RSS: 437927936
Load image 49
RSS: 437927936
RSS: 446222336
pop container 0
RSS: 446222336
RSS: 446222336
pop container 1
RSS: 446222336
RSS: 446222336
closed image in destructor
RSS: 446222336
RSS: 446226432
pop container 2
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 3
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 4
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 5
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 6
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 7
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 8
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 9
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 10
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 11
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 12
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 13
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 14
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 15
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 16
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 17
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 18
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 19
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 20
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 21
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 22
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 23
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 24
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 25
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 26
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 27
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 28
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 29
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 30
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 31
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 32
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 33
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 34
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 35
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 36
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 37
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 38
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 39
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 40
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 41
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 42
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 43
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 44
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 45
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 46
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 47
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 48
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 446226432
RSS: 446226432
pop container 49
RSS: 446226432
RSS: 446226432
closed image in destructor
RSS: 437927936
RSS: 437927936
Delete list
RSS: 437927936
RSS: 437927936
```
What are your OS, Python and Pillow versions?
  • OS: Ubuntu 20.04 (WSL2)
  • Python: Python 3.11.8
  • Pillow: 10.3.0
--------------------------------------------------------------------
Pillow 10.3.0
Python 3.11.8 (main, Feb 25 2024, 16:41:26) [GCC 9.4.0]
--------------------------------------------------------------------
Python executable is /home/kolbe/.cache/pypoetry/virtualenvs/tts-vtKShpo8-py3.11/bin/python3
Environment Python files loaded from /home/kolbe/.cache/pypoetry/virtualenvs/tts-vtKShpo8-py3.11
System Python files loaded from /usr
--------------------------------------------------------------------
Python Pillow modules loaded from /home/kolbe/.cache/pypoetry/virtualenvs/tts-vtKShpo8-py3.11/lib/python3.11/site-packages/PIL
Binary Pillow modules loaded from /home/kolbe/.cache/pypoetry/virtualenvs/tts-vtKShpo8-py3.11/lib/python3.11/site-packages/PIL
--------------------------------------------------------------------
--- PIL CORE support ok, compiled for 10.3.0
*** TKINTER support not installed
--- FREETYPE2 support ok, loaded 2.13.2
--- LITTLECMS2 support ok, loaded 2.16
--- WEBP support ok, loaded 1.3.2
--- WEBP Transparency support ok
--- WEBPMUX support ok
--- WEBP Animation support ok
--- JPEG support ok, compiled for libjpeg-turbo 3.0.2
--- OPENJPEG (JPEG2000) support ok, loaded 2.5.2
--- ZLIB (PNG/ZIP) support ok, loaded 1.2.11
--- LIBTIFF support ok, loaded 4.6.0
--- RAQM (Bidirectional Text) support ok, loaded 0.10.1, fribidi 1.0.8, harfbuzz 8.4.0
*** LIBIMAGEQUANT (Quantization method) support not installed
--- XCB (X protocol) support ok
--------------------------------------------------------------------
(tts-py3.11) kolbe@tt-ddm429-01:/mnt/c/dev/TTS$

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the supplied Python reproduction with PIL.Image, psutil, and the test_image.png input, then inspect the image copy and close behavior involved in Container._copyValue and Container.del. Compare memory readings while containers are popped and after garbage collection; done means copied image memory is released as containers are removed, or the issue is explained as allocator behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-vision
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.