python / python/cpython

test_dtrace should try to terminate SystemTap gracefully before SIGKILL

Open
#156,722 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

tests type-bug
Dominant language
Python
Stars
77.2k
Forks
35.9k
PR merge metrics
PR metrics pending

Description

Bug report

Bug description:

While checking out gh-156701 where the systemtap compilation was taking more than 10 seconds (hence an increase to timeout is required) I noticed also that the buildbot that I was testing at had too many D-state processes. That's possibly due to TraceBackend.trace() sending a SIGKILL immediately to the tracer process group when a timeout happens.

This prevents SystemTap and its children from performing normal cleanup. As mentioned in gh-156701 this left stap* temporary files behind, on the Fedora ppc64le buildbots, but also due repeated SystemTap timeouts, the buildbots had many orphaned staprun processes in Dstate, and the machines waited for them during shutdown.

From the system tap documentation: https://sourceware.org/systemtap/man/stap.1.html#abnormal-termination

One should avoid killing the stap process forcibly, for example with SIGKILL, because the stapio process (a child process of the stap process) and the loaded module may be left running on the system. If this happens, send SIGTERM or SIGINT to any remaining stapio processes, then use rmmod to unload the systemtap module.

Increasing the usability timeout should fix the premature timeout https://github.com/python/cpython/pull/156720, but genuine timeout still needs to be handled properly with SIGTERM and then if that fails a SIGKILL.

CPython versions tested on:

CPython main branch, 3.16, 3.15

Operating systems tested on:

Linux

Linked PRs
  • gh-156753

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with TraceBackend.trace() and the test_dtrace test mentioned in the report, then review how timeout handling reaches the SystemTap process group. Run the relevant DTrace/SystemTap tests and confirm that a genuine timeout allows graceful termination before forceful termination, without leaving orphaned processes or temporary files.

Written by the indexing model from the issue text.

Assessment

Tech stack
linux, python
Domain
operating-systems, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.