aio-libs / aio-libs/aiohttp

Growing memory usage in web_protocol.py

Đang mở
#10,671 1 bình luận 0 reaction 0 người được giao Xem trên GitHub
bug reproducer: missing
Ngôn ngữ chính
Python
Star
16.5k
Fork
2.4k
Merge trung bình
17 giờ 22 phút
Pull request đã merge (30 ngày)
212

Mô tả

### Describe the bug

I've been seeing a memory leak on one of our services, when running `tracemalloc`, one of the patterns I can see is that the memory usage in `web_protocol.py` keeps growing when being compared against the first snapshot. See below.

I'm wondering if it's a known issue that `web_protocol.py` isn't cleaning up ?

```
*** top 10 stats ***
/usr/local/lib/python3.10/site-packages/aiohttp/web_protocol.py:380: size=37.5 KiB (+2760 B), count=521 (+48), average=74 B
/usr/local/lib/python3.10/tracemalloc.py:423: size=1024 B (+1024 B), count=13 (+13), average=79 B
/usr/local/lib/python3.10/tracemalloc.py:560: size=608 B (+608 B), count=5 (+5), average=122 B
/usr/local/lib/python3.10/asyncio/base_events.py:438: size=1487 B (+504 B), count=12 (+3), average=124 B
/usr/local/lib/python3.10/tracemalloc.py:315: size=416 B (+416 B), count=8 (+8), average=52 B
/usr/local/lib/python3.10/site-packages/prometheus_client/multiprocess.py:148: size=2304 B (-336 B), count=12 (-2), average=192 B
/usr/local/lib/python3.10/tracemalloc.py:558: size=280 B (+224 B), count=5 (+4), average=56 B
/usr/local/lib/python3.10/site-packages/aiohttp/web_app.py:569: size=904 B (-168 B), count=2 (-1), average=452 B
/usr/local/lib/python3.10/abc.py:123: size=76.4 KiB (-63 B), count=936 (-1), average=84 B
/app/mtcnn_server/app/profiler.py:9: size=448 B (+32 B), count=2 (+1), average=224 B
*** top 10 stats ***
/usr/local/lib/python3.10/site-packages/aiohttp/web_protocol.py:380: size=38.2 KiB (+3450 B), count=533 (+60), average=73 B
/usr/local/lib/python3.10/tracemalloc.py:423: size=1176 B (+1176 B), count=16 (+16), average=74 B
/usr/local/lib/python3.10/tracemalloc.py:560: size=656 B (+656 B), count=6 (+6), average=109 B
/usr/local/lib/python3.10/tracemalloc.py:315: size=520 B (+520 B), count=10 (+10), average=52 B
/usr/local/lib/python3.10/site-packages/prometheus_client/multiprocess.py:148: size=2304 B (-336 B), count=12 (-2), average=192 B
/usr/local/lib/python3.10/asyncio/base_events.py:438: size=1319 B (+336 B), count=11 (+2), average=120 B
/usr/local/lib/python3.10/tracemalloc.py:558: size=336 B (+280 B), count=6 (+5), average=56 B
/app/mtcnn_server/app/profiler.py:9: size=480 B (+64 B), count=2 (+1), average=240 B
/usr/local/lib/python3.10/abc.py:123: size=76.4 KiB (-63 B), count=936 (-1), average=84 B
:672: size=85.2 KiB (+0 B), count=916 (+0), average=95 B
*** top 10 stats ***
/usr/local/lib/python3.10/site-packages/aiohttp/web_protocol.py:380: size=38.9 KiB (+4140 B), count=545 (+72), average=73 B
/usr/local/lib/python3.10/tracemalloc.py:423: size=1328 B (+1328 B), count=19 (+19), average=70 B
/usr/local/lib/python3.10/tracemalloc.py:560: size=704 B (+704 B), count=7 (+7), average=101 B
/usr/local/lib/python3.10/tracemalloc.py:315: size=624 B (+624 B), count=12 (+12), average=52 B
/usr/local/lib/python3.10/asyncio/base_events.py:438: size=1487 B (+504 B), count=12 (+3), average=124 B
/usr/local/lib/python3.10/site-packages/prometheus_client/multiprocess.py:148: size=2304 B (-336 B), count=12 (-2), average=192 B
/usr/local/lib/python3.10/tracemalloc.py:558: size=392 B (+336 B), count=7 (+6), average=56 B
/usr/local/lib/python3.10/site-packages/aiohttp/web_app.py:569: size=904 B (-168 B), count=2 (-1), average=452 B
/app/mtcnn_server/app/profiler.py:9: size=480 B (+64 B), count=2 (+1), average=240 B
/usr/local/lib/python3.10/abc.py:123: size=76.4 KiB (-63 B), count=936 (-1), average=84 B
*** top 10 stats ***
/usr/local/lib/python3.10/site-packages/aiohttp/web_protocol.py:380: size=39.5 KiB (+4830 B), count=557 (+84), average=73 B
/usr/local/lib/python3.10/tracemalloc.py:423: size=1480 B (+1480 B), count=22 (+22), average=67 B
/usr/local/lib/python3.10/tracemalloc.py:560: size=752 B (+752 B), count=8 (+8), average=94 B
/usr/local/lib/python3.10/tracemalloc.py:315: size=728 B (+728 B), count=14 (+14), average=52 B
/usr/local/lib/python3.10/asyncio/base_events.py:438: size=1487 B (+504 B), count=12 (+3), average=124 B
/usr/local/lib/python3.10/tracemalloc.py:558: size=448 B (+392 B), count=8 (+7), average=56 B
/usr/local/lib/python3.10/site-packages/prometheus_client/multiprocess.py:148: size=2304 B (-336 B), count=12 (-2), average=192 B
/usr/local/lib/python3.10/site-packages/aiohttp/web_app.py:569: size=904 B (-168 B), count=2 (-1), average=452 B
/app/mtcnn_server/app/profiler.py:9: size=480 B (+64 B), count=2 (+1), average=240 B
/usr/local/lib/python3.10/abc.py:123: size=76.4 KiB (-63 B), count=936 (-1), average=84 B
*** top 10 stats ***
/usr/local/lib/python3.10/site-packages/aiohttp/web_protocol.py:380: size=40.2 KiB (+5520 B), count=569 (+96), average=72 B
/usr/local/lib/python3.10/tracemalloc.py:423: size=1632 B (+1632 B), count=25 (+25), average=65 B
/usr/local/lib/python3.10/tracemalloc.py:315: size=832 B (+832 B), count=16 (+16), average=52 B
/usr/local/lib/python3.10/tracemalloc.py:560: size=800 B (+800 B), count=9 (+9), average=89 B
/usr/local/lib/python3.10/asyncio/base_events.py:438: size=1487 B (+504 B), count=12 (+3), average=124 B
/usr/local/lib/python3.10/tracemalloc.py:558: size=504 B (+448 B), count=9 (+8), average=56 B
/usr/local/lib/python3.10/site-packages/prometheus_client/multiprocess.py:148: size=2304 B (-336 B), count=12 (-2), average=192 B
/usr/local/lib/python3.10/site-packages/aiohttp/web_app.py:569: size=904 B (-168 B), count=2 (-1), average=452 B
/app/mtcnn_server/app/profiler.py:9: size=480 B (+64 B), count=2 (+1), average=240 B
/usr/local/lib/python3.10/abc.py:123: size=76.4 KiB (-63 B), count=936 (-1), average=84 B
*** top 10 stats ***
/usr/local/lib/python3.10/site-packages/aiohttp/web_protocol.py:380: size=40.9 KiB (+6210 B), count=581 (+108), average=72 B
/usr/local/lib/python3.10/tracemalloc.py:423: size=1784 B (+1784 B), count=28 (+28), average=64 B
/usr/local/lib/python3.10/tracemalloc.py:315: size=936 B (+936 B), count=18 (+18), average=52 B
/usr/local/lib/python3.10/tracemalloc.py:560: size=848 B (+848 B), count=10 (+10), average=85 B
/usr/local/lib/python3.10/asyncio/base_events.py:438: size=1487 B (+504 B), count=12 (+3), average=124 B
/usr/local/lib/python3.10/tracemalloc.py:558: size=560 B (+504 B), count=10 (+9), average=56 B
/usr/local/lib/python3.10/site-packages/prometheus_client/multiprocess.py:148: size=2304 B (-336 B), count=12 (-2), average=192 B
/usr/local/lib/python3.10/site-packages/aiohttp/web_app.py:569: size=904 B (-168 B), count=2 (-1), average=452 B
/app/mtcnn_server/app/profiler.py:9: size=544 B (+128 B), count=2 (+1), average=272 B
/usr/local/lib/python3.10/abc.py:123: size=76.4 KiB (-63 B), count=936 (-1), average=84 B
*** top 10 stats ***
/usr/local/lib/python3.10/linecache.py:137: size=529 KiB (+529 KiB), count=5311 (+5311), average=102 B
/usr/local/lib/python3.10/site-packages/aiohttp/web_protocol.py:380: size=42.2 KiB (+7590 B), count=605 (+132), average=71 B
/usr/local/lib/python3.10/tracemalloc.py:423: size=1936 B (+1936 B), count=31 (+31), average=62 B
/usr/local/lib/python3.10/sre_compile.py:804: size=3056 B (+1672 B), count=9 (+3), average=340 B
/usr/local/lib/python3.10/tracemalloc.py:535: size=1240 B (+1240 B), count=3 (+3), average=413 B
/usr/local/lib/python3.10/tracemalloc.py:315: size=1040 B (+1040 B), count=20 (+20), average=52 B
/usr/local/lib/python3.10/tracemalloc.py:447: size=896 B (+896 B), count=2 (+2), average=448 B
/usr/local/lib/python3.10/tracemalloc.py:248: size=832 B (+832 B), count=1 (+1), average=832 B
/app/mtcnn_server/app/profiler.py:38: size=832 B (+832 B), count=1 (+1), average=832 B
/usr/local/lib/python3.10/sre_parse.py:199: size=0 B (-832 B), count=0 (-1)
```

The traceback for that line is

```
File "/usr/local/lib/python3.10/site-packages/aiohttp/web_protocol.py", line 380
messages, upgraded, tail = self._request_parser.feed_data(data)
```

### To Reproduce

I think from a simple application like this will show the growing memory usage in `web_protocol.py`. I was just sending images to the `/image` enpdoint.

```
from aiohttp import web
import gc
import asyncio
import tracemalloc
from time import time

import objgraph

import tracemalloc

# list to store memory snapshots
snaps = []

def snapshot():
snaps.append(tracemalloc.take_snapshot())

def display_stats():
stats = snaps[0].statistics('filename')
print("\n*** top 5 stats grouped by filename ***")
for s in stats[:5]:
print(f"\n{str(s)}")

def compare():
first = snaps[0]
for snapshot in snaps[1:]:
stats = snapshot.compare_to(first, 'lineno')
print("\n*** top 10 stats ***")
for s in stats[:10]:
print(f"\n{str(s)}")

def print_trace():
# pick the last saved snapshot, filter noise
snapshot = snaps[-1].filter_traces((
tracemalloc.Filter(False, ""),
tracemalloc.Filter(False, ""),
tracemalloc.Filter(False, ""),
))
largest = snapshot.statistics("traceback")[0]

print(f"\n*** Trace for largest memory block - ({largest.count} blocks, {largest.size/1024} Kb) ***")
for largestt in largest.traceback.format():
print(f"\n{str(largestt)}")

async def hanlder(request):
print(f'read request')
req = await request.json()
return web.Response(text="Request has been receieved")

async def on_startup(app) -> None:
asyncio.create_task(show_memory())

async def show_memory():
print('start tracing memory')

tracemalloc.start(10)
start = tracemalloc.take_snapshot()

snapshot_num = 1
while True:
for _ in range(10):
await asyncio.sleep(2)

gc.collect()
snapshot()

display_stats()
compare()
print_trace()

my_web_app = web.Application()
my_web_app.router.add_route('POST', '/image', hanlder)
my_web_app.on_startup.append(on_startup)
web.run_app(my_web_app)
```

### Expected behavior

That the memory being used in `web_protocol.py` gets cleaned up.

### Logs/tracebacks

```python-traceback
File "/usr/local/lib/python3.10/site-packages/aiohttp/web_protocol.py", line 380
messages, upgraded, tail = self._request_parser.feed_data(data)
```

### Python Version

```console
$ 3.10.12
```

### aiohttp Version

```console
3.11.14
```

### multidict Version

```console
6.1.0
```

### propcache Version

```console
0.2.1
```

### yarl Version

```console
1.18.3
```

### OS

Ubuntu 22.04.5 LTS

### Related component

Server

### Additional context

_No response_

### Code of Conduct

- [x] I agree to follow the aio-libs Code of Conduct

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.