fork_processes() leaves child running without parent process after too many child restarts of other children

Open
#2,993 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
38/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
python
Domain
operating-systems

Research direction

Start at the tornado.process.fork_processes entry point and reproduce the sample with max_restarts=1, observing the parent and surviving child processes. Trace how the RuntimeError is handled after repeated child failures. Done means the working child is not orphaned when restart attempts are exhausted, while the give-up behavior is reported as intended.

Written by the indexing model from the issue text.

Description

process

I noticed that the parent process dies while leaving a child process alive if other children it tried to instantiate failed and max_attempts was reached. If there is still a working child, I would not expect the parent process to die and instead just stop trying to re-fork children once max_attempts is reached, perhaps logging a warning that it has given up.

Sample code:

import os

from tornado.ioloop import IOLoop
from tornado.netutil import bind_sockets
from tornado.process import fork_processes

sockets = bind_sockets(8888)
pid = os.getpid()
print(f"Parent PID: {pid}")
task_id = fork_processes(2, max_restarts=1)
pid = os.getpid()
print(f"Task ID for child PID {pid}: {task_id}")
if task_id == 0:
    IOLoop.current().start()
else:
    exit(1)

Logs:

$ python tornado_ioloop_test.py
Parent PID: 30431
Task ID for child PID 30436: 0
Task ID for child PID 30437: 1
child 1 (pid 30437) exited with status 1, restarting
Task ID for child PID 30438: 1
child 1 (pid 30438) exited with status 1, restarting
Traceback (most recent call last):
  File "tornado_ioloop_test.py", line 10, in <module>
    task_id = fork_processes(2, max_restarts=1)
  File "/home/user/miniconda3/envs/tornado_test/lib/python3.8/site-packages/tornado/process.py", line 167, in fork_processes
    raise RuntimeError("Too many child restarts, giving up")
RuntimeError: Too many child restarts, giving up
$ ps aux | grep tornado_ioloop_test
user 30436  0.0  0.0 134120 12656 pts/2    S    18:15   0:00 python tornado_ioloop_test.py

From the logs, you can see only the first child process (PID 30436) is left alive with both the parent process and failed child attempts dead.

Dominant language
Python
Stars
22.2k
Forks
5.6k
Avg merge
3h 42m
Merged PRs (30d)
16

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from tornadoweb/tornado

All issues in tornadoweb/tornado

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.