python-trio / python-trio/trio

shell command does not terminate on cancel

Open
#3,237 12 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
7.3k
Forks
431
Avg merge
2d 17h
Merged PRs (30d)
6

Description

As a simplified example, I would like to:

  • start a shell command
  • wait some time
  • stop it

The problem is that a child subprocesses from the shell won't terminate on cancel and would keep running even after python process is finished.

The problem is present only when a command consists of several processes, like ping localhost | cat, but not when a command is simple, like sleep 20.

Tested with: ubuntu-24.04, python 3.12.8, trio 0.29.0

Steps to reproduce:

  • check for currently running ping processes: ps aux | grep ping (confirm no ping process is running)
  • run the test script (see below) python ./test.py
  • check list of processes again, while the test script is running and after the test script terminates

Expected result is that a shell + ping + cat are all running only for 5 seconds.

Actual result that a shell process is running for 5 seconds, but the ping and also cat processes remain running not just beyond 5 seconds, but even after a script terminates. It looks like the subprocesses get detached from the shell and are not considered childs any more. I am not sure if this is 1 or 2 separate problems. I am not even sure if this is a proper way to run a shell script from within trio.

In any case, is there a workaround or a better way to run shell commands?

test.py script:

import trio
from subprocess import DEVNULL

# cmd = 'sleep 20'  # this is properly stopped
cmd = 'ping localhost | cat'  # this command won't stop properly

async def test():
    async with trio.open_nursery() as ns:
        async def service_runner(task_status = trio.TASK_STATUS_IGNORED):
            with trio.CancelScope() as scope:
                task_status.started(scope)
                await trio.run_process(cmd, shell=True, check=False,
                                        stdout=DEVNULL, stderr=DEVNULL)
                ns.cancel_scope.cancel()

        service_scope = await ns.start(service_runner)
        await trio.sleep(5.0)
        service_scope.cancel()
        await trio.sleep(5.0)

trio.run(test)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Run the supplied test.py with ping localhost | cat and compare its process list with the sleep 20 case. Then inspect the cancellation path around trio.run_process(..., shell=True); done means the shell and its child processes stop after cancellation and do not remain after the script exits.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.