MagicStack / MagicStack/uvloop

memory from transport buffer not freed after connection_lost

未关闭
#548 2 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

主要语言
Cython
星标
11.9k
派生
616
PR 合并指标
30 天内没有已合并 PR

描述

  • uvloop version: 0.17.0
  • Python version: 3.11.4
  • Platform: fedora 38
  • Can you reproduce the bug with PYTHONASYNCIODEBUG in env?: yes, with PYTHONASYNCIODEBUG=1
  • Does uvloop behave differently from vanilla asyncio? How?: yes, keeps resident memory

If I flood transport.write() and then close the connection (e.g. having the client telnet session stop) the memory stays resident.
Under stock asyncio, the memory immediately is released.

To reproduce,

  1. start this program,
  2. watch it in top
  3. , telnet to port 8888,
  4. watch memory usage spike.
  5. close the telnet session,
  6. see that memory usage stays high.
  7. repeat this experiment with stock asyncio runner,
  8. the memory is freed immediately. after telnet session is closed

This test script is a malloc bomb, so make sure you watch memory closely or you will consume all the memory on your machine

#!/usr/bin/env python3


import asyncio
import uvloop


class EchoServerProtocol(asyncio.Protocol):
    def __init__(self, on_connect, on_disconnect):
        self.on_connect = on_connect
        self.on_disconnect = on_disconnect

    def connection_lost(self, transport):
        print('Connection lost')
        self.on_disconnect()
        self.transport = None

    def connection_made(self, transport):
        self.on_connect(transport)
        peername = transport.get_extra_info('peername')
        print('Connection from {}'.format(peername))
        self.transport = transport

    def data_received(self, data):
        message = data.decode()
        print('Data received: {!r}'.format(message))

        print('Send: {!r}'.format(message))
        self.transport.write(data)

        print('Close the client socket')


async def send_data(transport):
    message = b'Hello World!' * 1000
    while True:
        transport.write(message)
        await asyncio.sleep(0)


task = None


def on_connect(transport):
    global task
    task = asyncio.create_task(send_data(transport))


def on_disconnect():
    global task
    task.cancel()


async def start():
    # Get a reference to the event loop as we plan to use
    # low-level APIs.
    loop = asyncio.get_running_loop()

    server = await loop.create_server(
        lambda: EchoServerProtocol(on_connect, on_disconnect),
        '127.0.0.1',
        8888,
    )

    async with server:
        await server.serve_forever()


if __name__ == '__main__':
    # with asyncio.Runner() as runner:
    with asyncio.Runner(loop_factory=uvloop.new_event_loop) as runner:
        runner.run(start())

贡献指南

这个仓库没有索引到贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

调研方向

首先使用 uvloop.new_event_loop 运行所提供的 Python 复现,先观察 telnet 会话关闭后的内存情况,然后将其与标准的 asyncio runner 进行比较。跟踪 transport buffer 和 connection_lost 路径;当断开连接后内存能够像在 asyncio 下那样被释放时,即视为完成。

由索引模型根据 Issue 内容生成。

评估

技术栈
python
领域
networking
Issue 类型
缺陷
难度
4/5
预计耗时
3-5 天
活跃度
停滞
描述清晰度
需要澄清
新手友好度
35/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。