tornadoweb / tornadoweb/tornado

Ping timeouts due to sequential frame processing

Open
#2,941 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

websocket
Dominant language
Python
Stars
22.2k
Forks
5.6k
Avg merge
3h 42m
Merged PRs (30d)
16

Description

We've been investigating a problem with our Tornado server, where the WebSocket connection would drop if the handling of a single message would take too much time (SO Question).

When we finally found that the reason for our disconnections was that the _receive_frame (Link) function in the WebSocket class processes only one message at a time, we replaced the awaiting of the result:

if is_final_frame:
    handled_future = self._handle_message(opcode, data)
    if handled_future is not None:
        await handled_future

With

if is_final_frame:
    handled_future = self._handle_message(opcode, data)
    if handled_future is not None:
        asyncio.create_task(handled_future)

With that change, everything seems to work great and we no longer have disconnections.

My questions are:

  1. Why is message handling not done in parallel to ping processing?
  2. What is the proper way of using lengthy handlers, without causing WebSocket ping timeouts?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with tornado/websocket.py at WebSocket._receive_frame and compare the awaited _handle_message path with the asyncio.create_task workaround described here. Review the linked Stack Overflow report for the timeout scenario. Done means establishing the intended relationship between lengthy message handlers and ping processing, then documenting or implementing the project-approved resolution.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.