python / python/cpython

Add runtime control for `profiling.sampling`

Open
#145,411 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

stdlib topic-profiling type-feature
Dominant language
Python
Stars
77.2k
Forks
35.9k
PR merge metrics
PR metrics pending

Description

Feature or enhancement

Proposal:

The idea is to add an optional control channel (socket? named pipe?), so another process can start, stop etc. profiling without killing/restarting the profiler completely. Right now, the only option is to interact with it via TUI mode or just stop it via Ctrl-C.

While working on https://github.com/psf/pyperf/pull/230 I ran into the following problem: some parts (eg: warming up) of benchmarking should not be profiled, as they do not add any value. My only option was sending SIGINT signal.

Generally, the current flow is nice for headful sessions but it could be improved for (especially long-running) headless sessions.

perf record --control does it in a similar way:

       --control=fifo:ctl-fifo[,ack-fifo], --control=fd:ctl-fd[,ack-fd]
           ctl-fifo / ack-fifo are opened and used as ctl-fd / ack-fd as
           follows. Listen on ctl-fd descriptor for command to control
           measurement.

       Available commands:

       •   enable : enable events
       •   disable : disable events
       •   enable name : enable event name
       •   disable name : disable event name
       •   snapshot : AUX area tracing snapshot).
       •   stop : stop perf record
       •   ping : ping
       •   'evlist [-v|-g|-F] : display all events
               -F  Show just the sample frequency used for each event.
               -v  Show all fields.
               -g  Show event group information.

Specifically:

  • We could start with just enable and disable initially. snapshot next but it'd be lovely to dump without loosing accumulated data. Or maybe even set_rate.
  • Possible transport layers could be Unix sockets (works great on Linux) or named pipe. One option might be portable, another not?
  • Possble new arguments like --control unix:/tmp/tachyon-ctl.sock or --control fifo:/tmp/tachyon-ctl, similar to what perf record --control does.

It could be also ommited from the documentation initially, so we don't limit ourselves too early.

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

No response

Linked PRs
  • gh-149958

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the existing profiling.sampling flow and how TUI mode and SIGINT currently control profiling. Compare the proposed enable/disable control flow with perf record --control, including the Unix socket or named-pipe options and the proposed --control argument. Done means a defined runtime-control interface that supports headless profiling without restarting the profiler.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
performance, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.